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4  Introduction 

This  project  is  aimed  at  improving  the  state  of  the  art  of  image-guided  and  minimally 
invasive  procedures  by  developing  a  new  generation  of  clinical  techniques  along  with  the 
computer-based  hardware  and  software  needed  for  their  implementation.  The  current 
focus  of  the  project  is  on  physician  assist  systems  incorporating  robotics,  tracking,  and 
visualization  to  improve  the  precision  of  instrument  placement  and  manipulation  in 
minimally  invasive  procedures.  The  project  is  led  by  the  Imaging  Sciences  and 
Information  Systems  (ISIS)  Center  of  the  Department  of  Radiology  at  Georgetown 
University.  Project  collaborators  include  the  Department  of  Radiation  Medicine  at 
Georgetown,  the  Urology  Robotics  Group  at  Johns  Hopkins  Medical  Institutions,  the 
NSF  sponsored  Engineering  Research  Center  for  Computer  Integrated  Surgical  Systems 
and  Technology  at  Johns  Hopkins  University,  and  the  Engineering  School  at  the  Catholic 
University  of  America. 

5  Report  Body 

This  section  describes  the  research  accomplishments  associated  with  each  task  in  the 
statement  of  work.  This  is  the  fifth  year  report  and  includes  research  performed  from  15 
January  2003  to  1  February  2004.  The  award  number  is  DAMD 17-99- 1-9022. 

5.1  Task  1:  Program  Planning  and  Management 

Program  planning  and  management  continues  to  focus  on  the  direction  of  the  project  as 
well  as  relationships  with  project  collaborators.  Project  planning  and  review  meetings  are 
held  monthly  at  the  ISIS  Center.  In  the  2003  calendar  year  major  progress  was  made  on 
several  projects  including  the  development  of  a  respiratory  motion  simulator  for  the 
CyberKnife  radiosurgery  system  and  new  work  on  robotic  Ivmg  biopsy.  We  also  have 
begun  planning  for  the  next  phase  of  the  project,  which  is  a  related  effort  on  the 
Operating  Room  of  the  Future. 

We  have  continued  our  very  close  cooperation  with  both  the  Urology  Robotics  Group  at 
Johns  Hopkins  Medical  Institutions  (lab  director  Dan  Stoianovici,  PhD)  and  the  NSF 
sponsored  Engineering  Research  Center  for  Computer  Integrated  Surgical  Systems  and 
Technology  at  Johns  Hopkins  University  (center  director  Russell  Taylor,  PhD). 

With  the  Engineering  Research  Center  at  Johns  Hopkins,  we  have  continued  to 
financially  support  a  PhD  student  (Sheng  Xu).  Mr.  Xu  has  been  working  on  the 
robotically  assisted  lung  biopsy  project  as  part  of  this  collaborative  effort.  Since  there  is 
no  Engineering  School  at  Georgetown  University,  this  provides  us  with  a  graduate 
student  to  help  develop  the  algorithms  and  software  for  this  testbed.  It  also  allows  us  to 
leverage  off  the  extensive  medical  robotics  program  at  Johns  Hopkins  University. 

The  group  continues  to  be  very  active  in  presenting  their  work  at  national  and 
international  conferences.  Dr.  Cleary  is  the  U.S.  program  committee  chair  for  the 
Computer  Assisted  Radiology  and  Surgery  conference,  which  will  be  held  in  June  2004 
in  Chicago.  Dr.  Cleary  organized  a  medical  robotics  workshop  for  the  June  2003  CARS 
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meeting.  He  also  organized  a  workshop  on  open  source  software  for  medical  devices  at 
the  November  2003  Medical  Image  Computing  and  Computer  Assisted  Interventions 
(MICCAI)  meeting.  The  research  group  also  had  two  papers  and  one  poster  accepted  at 
the  February  2003  SPIE  Medical  Imaging  Conference. 

5.2  Task  2:  Spinal  Robotics 

There  are  two  subtasks  under  this  category: 

1)  clinical  trial  of  “needle  driver”  robot  for  spinal  blocks 

2)  fluoroscopy  servoing  for  automatic  needle  positioning 

5.2.1  Clinical  Trial  of  “Needle  Driver”  Robot  for  Spinal  Blocks 

As  detailed  in  the  last  annual  report,  the  initial  clinical  trial  of  20  patients  using  the 
needle  driver  robot  for  spinal  blocks  was  conducted  at  Georgetown  University  fi'om 
August  2002  to  December  2002.  The  objective  of  this  trial  is  to  demonstrate  Aat  a 
physician  controlled  robotic  needle  driver  is  equivalent  in  safety  and  effectiveness  to  the 
standard  manual  technique  for  needle  placement  in  nerve  and  facet  blocks  in  the 
perispinal  region.  This  is  a  single  center,  randomized  feasibility  study  that  has  been 
approved  by  the  Georgetown  Institutional  Review  Board,  the  U.S.  Army  Human  Subjects 
Board,  and  the  Food  and  Drug  Administration  (FDA). 

The  results  have  since  been  tabulated  and  were  published  in  a  paper  at  the  Medical 
Imaging  Computing  and  Computer  Assisted  Interventions  meeting  (MICCAI)  in 
November  2003.  A  copy  of  the  paper  is  included  in  the  appendix  [Cleary  2003]’  and  the 
results  are  briefly  summarized  here.  A  picture  of  the  robot  is  shown  in  Figure  1  and  a 
picture  of  the  first  patient  is  shown  in  Figure  2. 


Mechanical 


Touch  screen 
and  — 
joystick  controls 

Mounting  base 


Figure  1:  Robotic  device  showing  mechanical  arm  and  joystick  control 

(courtesy  of  Dan  Stoianovici,  PhD,  Johns  Hopkins  Urology  Robotics) 


^  All  references  are  indicated  by  square  brackets  and  listed  in  the  reference  section  which  is  on  page  17. 
Copies  of  papers  are  in  the  appendices. 
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Figure  2:  Clinical  trial  of  robotic  device  for  nerve  and  facet  blocks  at  Georgetown 
University  (interventional  radiologist  is  Vance  Watson,  MD) 


The  clinical  trial  showed  that  is  feasible  to  use  a  joystick  controlled  robot  for  nerve  and 
facet  blocks.  While  this  was  a  pilot  study  and  not  enough  data  was  gathered  for  statistical 
significance,  some  general  trends  can  be  observed.  The  mean  accuracy  in  the  robot  (1.105 
mm)  and  manual  (1 .238  mm)  is  about  the  same.  Therefore,  it  appears  that  the  robot  is 
capable  of  accurate  needle  placement. 

As  expected,  the  pain  score  post-treatment  was  significantly  less  than  the  pain  score  pre¬ 
treatment  in  both  the  robot  and  manual  arms.  In  the  robot  arm,  pain  scores  fell  from  a 
mean  of  6.3  pre-treatment  to  1 .8  post-treatment.  In  the  manual  arm,  pain  scores  fell  fi'om 
6.0  pre-treatment  to  0.9  post-treatment.  Patients  had  to  sign  an  informed  consent  form 
and  were  generally  receptive  to  the  use  of  the  robot. 

Based  on  the  success  of  this  initial  trial,  we  applied  to  the  FDA  to  continue  the  clinical 
trial  with  an  additional  80  patients.  The  FDA  has  granted  approval  for  this  and  we  are 
now  working  with  the  Georgetown  and  Army  boards  for  their  approval  as  well. 

5.2.2  Fluoroscopy  Servoing  for  Automatic  Needle  Positioning 

The  goal  of  this  work  is  to  develop  an  algorithm  for  automatic  needle  placement  with  the 
robot  during  minimally  invasive  spine  procedures.  This  automatic  needle  placement  is 
achieved  by  a  closed-loop  position  control  of  the  robot  end-effector  provided  by  a 
computer  algorithm.  The  term  fluoroscopy  servoing  is  given  to  this  process  since  an  x- 
ray  fluoroscope  is  used  as  the  visualization  tool.  This  work  was  done  by  Gabriela  Corral 
of  The  Catholic  University  of  America,  who  completed  her  master’s  thesis  on  this  topic 
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in  January  2004.  A  key  contributor  to  this  work  was  Luis  Ibanez,  PhD,  of  Kitware 
Incorporated.  An  extended  abstract  describing  this  work  was  accepted  for  presentation  at 
the  Computer  Assisted  Radiology  and  Surgery  Meeting  in  Chicago  in  June  2004  [Corral 
2004]. 

Figure  3  is  a  flow  diagram  of  the  procedure  used  for  tracking  the  needle  on  the 
fluoroscopic  image.  First,  the  image  is  acquired  using  the  digital  frame  grabber  card. 
Using  this  initial  image  as  input,  a  robust  but  relatively  slow  process  is  applied  for 
performing  an  initial  detection  of  the  needle.  The  output  of  this  process  is  a  list  of  pixel 
coordinates  belonging  to  the  needle.  From  this  set  of  points,  a  number  of  heuristics  are 
applied  in  order  to  identify  the  tip  and  head  of  the  needle.  Once  the  initial  detection  is 
completed  the  robot  will  move  and  the  needle  will  be  detected.  This  is  done  until  enough 
information  is  gathered  to  get  the  transformation  from  image  to  robot  space  to  move  the 
robot  to  the  desire  target  point. 

Start 


i 


Figure  3:  Flow  diagram  of  needle  tracking  process 


A  cadaver  study  was  performed  to  test  the  accuracy  of  this  application.  Twenty  trials 
were  performed.  The  difference  in  errors  between  the  manual  trials  and  the  automatic 
trials  was  very  small.  On  average  the  measured  root  mean  square  error  for  the  manual 
trials  was  1 .39  +/- 1 .02mm  and  1 .2 1  +/-0.66mm  for  the  automatic  trials.  Both  of  these  are 
clinically  acceptable.  We  plan  to  continue  these  developments,  with  the  eventual  goal  of 
introducing  these  algorithms  into  clinical  practice. 
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5.3  Task  3:  Robot  Biopsy  Testbed  and  Lung  Biopsy 

The  goals  of  the  robot  biopsy  testbed  were: 

1 .  to  compare  robotically  assisted  biopsy  to  the  current  practice 

2.  serve  as  a  testbed  for  investigating  software  architectures  for  integrating  robotics, 
tracking,  and  visualization. 

The  initial  feasibility  study  was  completed  on  the  testbed  during  the  last  report  period  and 
paper  describing  the  study  was  submitted  to  a  special  issue  on  Medical  Robotics  of  the 
journal  IEEE  Transactions  on  Robotics  and  Automation. 

While  this  study  is  complete,  we  have  embarked  on  a  related  effort  here  on  robotically 
assisted  lung  biopsy.  This  effort  is  also  partially  funded  by  the  National  Cancer  Institute 
under  grant  1  R21  CA94274-01A1.  The  goal  of  this  effort  is  to  demonstrate  the 
feasibility  of  robotically  assisted  lung  biopsy  under  CT  fluoroscopy  [Xu  2004]. 

In  lung  biopsy,  the  position  of  the  target  lesion  may  vary  due  to  intrinsic  causes  such  as 
respiratory  motion  or  extrinsic  reasons  such  as  interactions  between  the  tissue  and  a 
surgical  tool.  In  developing  a  robotic  system  to  assist  in  biopsy,  we  need  to  provide  some 
way  to  track  the  target  lesion.  CT  fluoroscopy  combines  the  advantages  of  both  CT  and 
fluoroscopy,  which  offers  an  opportunity  to  track  the  motion  of  the  target  lesion  in  real¬ 
time  during  the  intervention.  Our  prototype  system  is  shown  in  Figure  4.  The  real-time 
CT  fluoroscopy  image  is  captured  using  a  frame  grabber  (Accustream  170,  Foresight 
Imaging,  Lowell,  Massachusetts,  USA).  The  position  of  the  target  lesion  is  detected  from 
the  image,  which  serves  as  input  for  the  robot  controller  to  compensate  for  the  motion. 

We  have  developed  a  motion  analysis  method  to  estimate  the  3D  motion  of  a  target  lesion 
using  the  CT  fluoroscopy  image  sequences.  The  method  aims  at  tracking  the  lesion’s 
motion  no  matter  whether  the  lesion  is  inside  or  outside  the  imaging  plane. 


Figure  4:  CT  fluoroscopy  guided  robotically  assisted  lung  biopsy 


Three  experiments  were  carried  out  using  both  synthetic  and  real  CT  fluoroscopy  images 
(Siemens  Somatom  Volume  Zoom  CT).  The  synthetic  CT  fluoroscopy  image  was 
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generated  from  a  pre-operative  CT  volume  of  the  lung  region  with  2.0  mm  slice  thickness 
and  0.72  mm  pixel  size.  As  a  result,  the  exact  match  of  the  CT  fluoroscopy  image  region 
is  known.  It  was  observed  that,  for  the  objective  function  to  converge,  the  starting  point 
had  to  be  vdthin  +2mm  for  translation  and  within  ±20°  for  each  of  the  three  rotation 
angles.  A  similar  experiment  was  made  using  two  different  CT  volumes  of  the  same 
patient.  With  one  volume  as  the  preoperative  CT  volume,  and  the  other  one  to  generate 
synthetic  CT  fluoroscopy  images,  similar  results  were  obtained. 


The  second  experiment  was  performed  with  CT  fluoroscopy  images  from  a  respiratory 
motion  simulator.  As  shown  in  Figure  5,  the  target  organ  had  one  degree  of  freedom  and 
was  moved  by  a  motor  in  the  cranio-caudal  direction.  After  a  preoperative  volumetric 
scan  was  obtained,  the  data  was  saved  in  DICOM  format  with  512x512  resolution,  1 
mm  slice  thickness  and  0.74  mm  pixel  size.  The  motor  of  the  simulator  was  aligned  with 
the  CT  gantry  such  that  the  motion  of  the  phantom  was  perpendicular  to  the  imaging 
plane  (Figure  6).  Since  flie  organ  phantom  was  rigid,  its  motion  curve  was  the  same  as 
that  of  the  motor.  A  sequence  of  CT  fluoroscopy  images  were  taken  at  about  6Hz  and 
saved  in  the  DICOM  format.  The  resolution  of  the  CT  fluoroscopy  image  was  256  x  256, 
with  lOmm  slice  thickness  and  1.48  mm  pixel  size.  The  tull  details  are  given  in  the  paper 
[Xu  2004],  but  the  tracking  results  showed  an  average  position  error  of  under  1  mm.  The 
third  experiment  was  a  swine  study  and  is  also  described  in  the  paper. 


Figure  5:  Respiratory  motion  simulator 


Figure  6:  Motion  tracking  test  of 
respiratory  motion  simulator 


5.4  Task  4:  Liver  Respiratory  Motion  Simulator  and 
Internal  Organ  Tracking 

The  goal  of  this  task  is  to  investigate  the  use  of  electromagnetic  tracking  for  precisely 
locating  internal  organs  such  as  the  liver  during  interventional  procedures.  This  is  a 
collaboration  with  Northern  Digital  (Waterloo,  Canada)  and  Traxtal  Technologies 
(Houston,  Texas).  Northern  Digital  has  developed  the  AURORA™  electromagnetic 
tracking  system,  which  enables  instruments  that  are  fitted  with  a  sensing  coil  to  be 
tracked  and  overlaid  on  an  image  of  the  anatomy.  Our  research  group  at  Georgetown 
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served  as  a  beta  test  site  and  was  one  of  the  first  research  groups  worldwide  to  receive 
this  equipment.  The  system  has  recently  been  released  as  a  commercial  product  as  shown 
in  Figure  7. 


Figure  7:  AURORA™  sensors,  electromagnetic  tracking  system  components, 

and  measurement  volume 

The  left  picture  shows  (from  left  to  right)  the  control  imit,  sensor  interface  device,  and  electromagnetic 
field  generator.  The  middle  picture  shows  the  sensor  coils  along  with  the  electrical  wires  protruding  from 
the  coil,  compared  to  a  match.  The  right  picture  shows  the  measurement  volume  in  mm  relative  to  the 
location  of  the  field  generator.  (Photos  courtesy  of  Northern  Digital,  Inc.) 


As  part  of  this  research,  we  have  developed  an  image-guided  surgery  system  [Tang 
2003].  In  this  system,  registration  is  done  based  on  pre-operative  CT  scans  and  a  set  of 
fiducials  that  can  be  localized  in  both  image  and  electromagnetic  space.  The  AURORA 
uses  small  sensor  coils  that  can  be  embedded  into  the  tip  of  needles  or  catheters  to 
directly  track  their  position  and  orientation.  The  system  allows  the  physieian  to 
accurately  guide  a  magnetically  tracked  needle  into  the  abdominal  area  for  a  variety  of 
interventional  procedures  that  are  usually  done  xmder  x-ray  fluoroscopy.  The  system  has 
been  tested  on  phantom  torsos,  respiratory  motion  simulators,  and  cadavers,  all  of  which 
have  been  rigid-body  approximations  of  true  physiological  conditions.  In  a  liver 
phantom,  the  physician  using  the  system  has  been  able  to  target  simulated  liver  tumors 
with  successful  placement  in  14  of  16  attempts.  However,  the  system  does  not  perform 
as  well  with  targets  smaller  than  1cm,  deeper  targets,  and  various  other  experimental 
conditions.  This  prompted  us  to  investigate  the  error  sources  of  the  system. 


Figure  8  shows  the  implementation  of  the  system  as  recently  used  in  an  approved  swine 
study  in  the  interventional  suite.  The  goal  of  this  study  was  to  hit  simulated  lesions  in  a 
swine  liver  using  pre-procedure  CT  scans  and  image  guidance  based  on  electromagnetic 
tracking.  The  physieian  watches  the  image  overlay  shown  in  the  lower  left  of  the  figure  to 
manipulate  the  needle  along  a  pre-planned  path.  Respiration  is  accounted  for  using 
another  electromagnetically  tracked  needle  that  was  placed  in  the  liver  prior  to  the  CT 
scan.  The  current  system  had  mixed  success  in  aceurately  hitting  the  targets,  whieh 
prompted  us  to  investigate  the  sources  of  error  in  the  system. 
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Figure  8:  Swine  study  in  the  interventional  suite 
(interventional  radiologist  is  Elliot  Levy,  MD) 

The  main  sources  of  error  in  our  rigid  model  include  errors  in  electromagnetic  space, 
image  space,  and  human  error  in  physical  space.  Figure  9  below  depicts  the  propagation 
of  error  from  electromagnetic  and  image  space  to  fiducial  registration  and  target 
registration. 


Figure  9:  Accuracy  breakdown  of  our  registration  process 

We  categorized  the  error  sources  into  three  main  groups:  1)  electromagnetic  space,  2)  CT 
space,  and  3)  physical  space. 

1)  Electromagnetic  space.  The  most  obvious  error  source  is  the  inherent  accuracy  of  the 
electromagnetic  tracking  system.  We  use  circular  multi-modality  fiducials  (IZI 
Corporation,  Baltimore,  MD)  as  skin  fiducials  in  our  registration  process.  For  the 
experiment  described  in  this  paper,  we  built  a  custom  plexiglass  registration  phantom 
using  0.8mm  tantalum  beads  (Tilly  Medical,  Sweden)  as  fiducials.  To  localize  a  fiducial 
in  electromagnetic  space,  the  center  of  the  fiducial  is  touched  with  a  magnetically  tracked 
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probe.  Error  sources  here  include  the  placement  error  in  manually  localizing  the  center 
of  the  fiducial  and  error  in  calculating  the  location  of  the  probe  tip  from  the  embedded 
sensor  coil,  which  involves  the  combined  translational  and  orientation  error  of  the 
tracking  system. 

2)  CT  space.  There  will  be  some  error  inherent  to  the  CT  scanner  and  image 
reconstruction  process  but  this  is  considered  to  be  fairly  small  as  we  are  using  1  mm 
slices  in  our  experimental  studies.  Most  of  the  error  in  localizing  image  fiducials  will 
result  from  our  manual  process  of  choosing  the  center  of  the  fiducials  on  the  CT  scan. 

3)  Physical  space.  In  any  registration  procedure  where  manual  intervention  is  required  to 
select  or  localize  the  fiducials,  there  will  always  be  human  error.  In  our  system,  human 
error  exists  at  the  step  where  we  localize  the  skin  fiducials  in  electromagnetic  space  and 
image  space;  when  the  user  taps  out  the  centers  of  the  skin  fiducials  on  the  patient  and 
clicks  on  the  center  of  the  image  fiducials  on  the  CT  scan.  This  error  can  not  be  directly 
measured  independently  from  errors  in  the  rest  of  our  system.  Errors  in  electromagnetic, 
image  and  physical  space  influenced  the  system’s  ability  to  accurately  localize  fiducials. 

The  results  of  this  error  analysis  are  detailed  in  the  paper  [Tang  2003].  There  are  many 
factors  that  can  contribute  to  the  overall  accuracy  of  an  image-guided  surgery  system 
based  on  electromagnetic  tracking.  A  systematic  analysis  is  necessary  to  isolate  critical 
factors  and  determine  how  optimal  performance  can  be  achieved.  We  believe  that  the 
greatest  contributor  to  registration  error  is  from  the  manual  fiducial  localization  process. 
Our  simulation  indicated  that  if  we  were  to  have  a  system  with  automatic  fiducial 
localization,  we  would  obtain  much  closer  fiducial  registration.  This  is  now  the  focus  of 
current  research. 

5.5  Task  5:  Radiation  Treatment 

(Stereotactic  Radiosurgery) 

This  research  is  being  carried  out  in  conjunction  with 
the  Radiation  Medicine  Department  at  Georgetown.  A 
new  system  for  precision  radiation  therapy  (the 
CyberKnife)  was  installed  at  Georgetown  in  Spring 
2002.  The  system  includes  a  robotic  arm  which  is 
capable  of  precisely  positioning  the  radiation  beam 
and  moving  it  in  real-time  to  compensate  for  organ 
motion  due  to  respiration.  Our  joint  research  project  is 
to  investigate  the  use  of  skin  motion  as  a  predictor  of 
internal  organ  motion  to  enhance  the  precision  of 
these  treatments.  A  protocol  to  collect  this  data  during 
ongoing  patient  treatments  has  been  approved  by  the 
Georgetown  Institutional  Review  Board  and  the 
Army  Human  Subjects  Research  Review  Board.  A 
picture  of  the  test  setup  is  shown  in  Figure  10. 
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The  results  from  the  first  four  patients  are  given  in  Table  1  [Dieterich  2003a].  Two  of  the 
patients  were  treated  in  three  fractions  (designated  hy  Fx  in  the  table).  Four  markers  were 
placed  on  the  first  three  patients,  and  three  markers  on  the  fourth  patient.  Movement 
amplitudes  of  individual  markers  ranged  from  3.1  mm  to  14.8  mm  with  a  median  of  7.5 
mm;  the  centroid  amplitudes  were  between  2.1  mm  and  6.0  mm.  The  average  movement 
of  the  skin  marker  centroid  was  4.7  mm,  3.3  mm,  2.6  mm  and  6.0  mm. 


Table  1 :  Skin  marker  motion  for  four  patients 


Fx 

#2  (mm) 

WdSsSSM 

#4  (mm) 

Patient  1 

1 

6.9 

7.6 

7.7 

N/A 

4.7 

Patient  2 

1 

4.1 

6.6 

5.9 

4.4 

3.7 

2 

8.0 

12.2 

10.9 

6.2 

3.4 

3 

8.1 

12.0 

11.6 

l.S 

2.9 

Patient  3 

1 

6.3 

7.7 

8.5 

5.6 

2.1 

2 

6.5 

5.8 

7.0 

4.2 

2.3 

3 

14.8 

14.2 

12.9 

13.5 

3.3 

Patient  4 

1 

9.9 

5.1 

3.1 

N/A 

6.0 

The  centroids  of  the  internal  fiducial  markers  were  extracted  from  the  CyberKnife  log 
files.  The  centroids  moved  2.5  mm  and  1 .7  mm  for  the  two  patients  with  fiducials  placed 
in  the  sacrum.  The  fiducial  markers  in  the  pancreas  moved  3.5  mm,  and  2.0  mm  in  the 
lung.  There  were  no  statistical  differences  in  either  external  or  internal  marker 
movements  for  the  first,  second  and  third  part  of  one  treatment. 

The  skin  motion  data  was  then  analyzed  using  the  Fast  Fourier  Transformation  from 
Matlab.  Breathing  patterns  for  a  particular  patient  remain  consistent  over  several 
treatment  stages.  Breathing  patterns  vary  widely  between  patients;  some  individuals  have 
a  very  consistent  and  predictable  breathing,  others  show  irregular  breathing  patterns.  In 
general,  breathing  frequencies  ranged  from  13  Hz  to  23  Hz. 

Figure  1 1  shows  the  motion  of  the  skin  marker  centroid  of  Patient  4  of  our  sample  with 
the  corresponding  FFT  analysis.  The  patient  was  diagnosed  with  a  bronchoalveolar 
carcinoma  of  the  lung  and  underwent  multiple  thoracic  surgeries  including  upper 
lobectomy,  right  middle  lobectomy,  a  left  upper  lobectomy,  and  right  lower  wedge 
resection.  She  was  wearing  nasal  oxygen  and  experienced  some  anxiety  during  the 
treatment.  While  her  breathing  cycle  is  fairly  regular,  two  distinct  features  could  be  seen 
from  the  figure.  The  first  feature  was  the  very  high  breathing  rate  at  about  21  breaths  per 
minute.  Second,  every  few  breaths  she  took  a  longer  inhalation,  which  also  shows  two 
local  maxima.  Three  examples  of  this  can  be  seen  in  the  figure  at  38  s,  68  s  and  78  s  after 
the  start  of  the  recording. 
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Figure  11:  Motion  of  skin  marker  centroid  of  patient  4  (left)  and  FFT  analysis  (right) 

It  was  found  that  great  care  needed  to  be  taken  in  placing  the  skin  markers  and  in 
arranging  the  environment.  For  example,  items  that  reflect  light  such  as  belt  buckles  or 
shirt  buttons  can  cause  crosstalk,  i.e.,  five  instead  of  four  skin  markers  were  recognized. 
Similar  problems  would  occur  if  skin  markers  were  placed  too  close  together.  Therefore, 
patients  were  asked  to  wear  a  hospital  gown  for  their  treatment,  and  the  markers  were 
placed  at  least  10  cm  apart.  We  also  did  a  software  cut  on  markers  more  than  one  meter 
away  from  the  expected  direction.  The  marker  placement  also  was  varied  according  to  the 
patient’s  position  on  the  treatment  table  (head  first  supine  or  feet  first  supine).  For  feet 
first  supine  patients,  the  markers  were  placed  more  inferiorly  on  the  abdomen. 

Finally,  this  work  convinced  us  of  the  need  to  develop  a  respiratory  motion  simulator  for 
the  CyberKnife  environment.  This  simulator  will  be  used  to  study  potential  correlations 
between  skin  motion  and  internal  organ  motion  and  whether  skin  motion  can  be  used  as  a 
predictor  of  internal  organ  motion  for  stereotactic  radiosurgery.  The  simulator  was 
designed  by  a  local  consultant,  Tong  Zhou,  PhD,  using  computer  aided  design  software 
as  shown  in  Figure  12.  The  simulator  was  constructed  in-house  and  the  initial  testing  was 
done  in  the  CyberKnife  suite  as  shown  in  Figure  1 3.  A  paper  was  written  describing  the 
simulator  which  tied  for  first  prize  at  the  annual  CyberKnife  User’s  Ciroup  meeting 
[Dieterich  2003b]. 
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5.6  Year  6  Plans 

In  year  6,  we  plan  to  continue  our  research  in  three  major  areas: 

1)  robotic  assistance  for  needle  placement  in  the  spine  and  lung 

2)  electromagnetic  position  sensing  and  guidance  for  abdominal  interventions 

3)  skin  motion  tracking  for  stereotactic  radiosurgery  and  correspondence  to  internal 
organ  motion 

Our  overall  goal  remains  the  same:  technology  development  to  provide  assistance  to  the 
physician  during  minimally  invasive  interventions.  One  of  our  project  themes  has  been 
the  characterization  of  respiratory  motion  and  we  will  continue  to  do  studies  in  this  area. 
We  will  also  continue  to  look  for  new  funding  opportunities  and  S5mergistic 
collaborations. 


6  Key  Research  Outcomes 

This  section  provides  a  bulleted  list  of  key  research  accomplishments: 

•  Received  FDA  approval  to  continue  the  clinical  trial  for  robotically  assisted  spinal 
blocks  with  an  additional  80  patients 

•  Demonstrated  the  feasibility  of  fluoroscopy  servoing  for  automatically  positioning 
a  needle  driver  robot  at  the  skin  entry  point 

•  Began  a  new  study  of  robotically  assisted  lung  biopsy  and  demonstrated  the 
ability  to  frame  grab  CT  fluoroscopy  images  and  display  them  on  the  robot 
controller 

•  Completed  an  initial  set  of  approved  animal  studies  demonstrating  the  feasibility 
of  electromagnetic  position  sensing  for  image  guidance  in  the  interventional  suite 

•  Gathered  skin  motion  data  on  four  patients  during  CyherKnife  stereotactic 
radiosurgery  treatments  imder  an  Il^-approved  protocol 

•  Develop  a  respiratory  motion  simulator  consisting  of  two  3  degree  of  freedom 
platforms  to  evaluate  CyberKnife  treatments  which  compensate  for  internal  organ 
motion 

7  Reportable  Outcomes 

This  section  provides  a  list  of  reportable  outcomes. 

The  major  product  of  this  year  is  the  list  of  papers  given  in  Section  9,  References.  Copies 

of  these  documents  are  provided  in  the  appendix. 

In  addition,  several  grant  applications  to  the  National  Institutes  of  Health  were  submitted 

based  on  this  work.  Two  small  business  Phase  I  grant  applications  were  funded  in  2003: 

1)  the  first  one  was  a  STTR  grant  from  NIH/NIBIB  to  Kitware  Inc.  to  develop  a  software 
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toolkit  for  image-guided  surgery  and  2)  the  second  one  was  a  SBIR  grant  from 
NIH/NHLBI  to  Traxtal  Technologies  to  develop  electromagnetically  tracked  bone 
screws.  On  both  of  these  grants,  subcontracts  were  issued  to  Georgetown  University  for 
part  of  the  research. 

A  graduate  student  from  Catholic  University  and  a  graduate  student  from  Johns  Hopkins 
University  were  supported  during  the  year  to  assist  in  software  development  for  the 
robotic  needle  driver.  The  research  group  at  Georgetown  continued  to  take  a  lead  in  the 
Washington  Area  Computer  Aided  Surgery  Society  (www.washcas.orgJ.  which  was 
formed  in  2000  to  promote  research  in  the  field. 

8  Conclusions 

The  fifth  year  of  work  on  the  Periscopic  Spine  Surgery  has  continued  to  lay  the 
groundwork  for  developing  the  physician  assist  systems  of  the  future.  These  systems  will 
incorporate  robotics,  tracking,  and  visualization  to  improve  the  precision  of  instrument 
placement  and  manipulation  in  minimally  invasive  procedures.  Approval  was  received 
from  the  FDA  to  continue  the  robotic  spinal  blocks  trial  with  an  additional  80  patients. 
The  feasibility  of  electromagnetic  position  sensing  and  image  overlay  was  demonstrated 
in  an  animal  study.  A  respiratory  motion  simulator  was  developed  to  evaluate 
CyberKnife  treatments  which  compensate  for  internal  organ  motion.  The  focus  of  the 
next  year  will  continue  to  be  on  technology  improvements  and  moving  this  technology  to 
clinical  practice  to  improve  patient  care. 
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10  Appendices  (Papers) 


Copies  of  the  eight  papers  published  during  this  report  period  are  reproduced  in  this 
section. 

10.1  Cleary  2003:  Robotically  Assisted  Interventions  ... 

Reprint  begins  on  the  next  page  and  is  2  pages. 
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Robotically  Assisted  Interventions: 

Clinical  Trial  for  Spinal  Blocks 

Kevin  Cleary^,  Vance  Watson*,  David  Lindisch*, 

Alexandra  Patricia^,  Dumitru  Mazilu^,  and  Dan  Stoianovici^ 

*  Imaging  Science  and  Information  Systems  (ISIS)  Center,  Department  of  Radiology, 
Georgetown  University  Hospital,  Washington,  DC,  USA 
{ clearyQgeorgetown . edu } 

^  Urobotics  Laboratory,  Johns  Hopkins  Medical  Institutions,  Baltimore,  MD,  USA 


Abstract  Percutaneous  interventions  are  performed  by  freehand  passages  of  in¬ 
struments,  such  as  needles,  from  the  skin  surface  to  the  anatomy  of  interest.  The 
main  problem  with  this  approach  is  that  the  physician  can  be  inaccurate  in 
aligning  the  instrument  and  staying  on  course.  A  joystick-controlled  robotic 
needle  driver  may  allow  the  physician  to  more  precisely  target  the  anatomy. 
This  paper  describes  our  experience  with  a  robotic  needle  driver  in  a  20-patient 
clinical  trial  of  nerve  and  facet  blocks.  Our  next  stage  of  research  in  robotically 
assisted  lung  biopsy  is  also  described. 


1  Materials  and  Methods 

The  robotic  needle  driver  consists  of  a  three  degree-of-freedom  (DOF)  translational 
stage,  a  2-DOF  rotational  stage  which  can  orient  the  needle  to  any  angle,  and  a  1-DOF 
needle  drive  mechanism  [1].  The  robot  is  controlled  using  a  joystick  and  touch  screen. 
The  interventionalist  can  thus  manipulate  the  needle  under  x-ray  fluoroscopy  without 
direct  exposure  to  the  radiation  beam.  After  cadaver  studies  using  the  robot  to  pre¬ 
cisely  position  a  needle  in  the  lumbar  spine  were  successfully  completed  [2],  a  ran¬ 
domized  clinical  trial  of  20  patients  undergoing  nerve  and  facet  blocks  was  approved 
by  the  FDA  and  the  local  institutional  review  board.  The  procedure  is  done  in  the 
standard  manner  except  the  robot  is  used  to  position,  orient,  and  drive  the  needle  un¬ 
der  physician  control.  A/P  fluoroscopy  is  used  to  position  and  orient  the  needle,  and 
lateral  fluoroscopy  is  used  to  monitor  the  depth  of  insertion. 

The  robot  is  mounted  on  the  interventional  table  using  a  custom-designed  locking 
mechanism.  The  robot  is  positioned  initially  near  the  skin  entry  point  by  loosening  the 
passive  gross  positioning  mechanism  and  moving  the  needle  driver  end  of  the  robot  by 
hand.  Once  this  initial  position  has  been  attained,  the  mechanism  is  locked  and  the 
robot  is  switched  to  operate  by  physician  control. 

2  Results 

The  study  was  approved  by  the  local  Institutional  Review  Board  and  the  Food  and 
Drug  Administration  (FDA)  and  was  conducted  from  August  to  December,  2002.  The 
study  was  completed  by  a  single  fellowship  trained  interventional  neuroradiologist  at 
Georgetown  University  Hospital  using  a  Siemens  Neurostar  bi-plane  fluoroscopy 
system.  The  standard  manual  technique  was  used  on  10  patients  and  the  robotic  device 
was  used  on  10  patients.  The  patients  ranged  in  age  from  30  to  70  years.  The  spine 
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levels  were  from  S-1  to  L-5.  No  complications  were  observed  in  the  study.  One  of  the 
patients  in  the  robotics  arm  had  to  be  converted  to  a  manual  procedure  due  to  slippage 
of  the  needle  driver.  This  conversion  was  done  vdthout  difficulty  or  complications. 

There  were  two  outcome  measures:  1)  accuracy  of  needle  placement  and  2)  pain  re¬ 
lief  Accuracy  of  needle  placement  was  determined  as  follows.  Before  the  interven- 
tionalist  began  placing  the  needle,  both  an  A/P  and  lateral  image  of  the  patient  were 
obtained.  The  interventionalist  would  then  annotate  each  image  with  an  arrow  to  indi¬ 
cate  the  desired  target  location  of  the  needle  (the  interventionalist  was  not  blinded  as 
to  manual/robotic  technique  as  this  was  not  practical).  After  the  needle  was  placed,  an 
A/P  and  lateral  image  was  again  obtained.  The  two  sets  of  images  were  compared  to 
determine  the  distance  between  the  intended  location  of  the  needle  and  the  actual 
location  of  the  needle.  Pain  relief  was  measured  using  a  visual-analog  scale,  with  0 
representing  no  pain  and  10  representing  excruciating  pain. 

3  Discussion 

The  results  to  date  show  that  is  feasible  to  use  a  joystick  controlled  robot  for  nerve 
and  facet  blocks.  While  this  was  a  pilot  study  and  not  enough  data  was  gathered  for 
statistical  significance,  some  general  trends  can  be  observed.  The  mean  accuracy  in 
the  robot  (1.105  mm)  and  manual  (1.238  mm)  is  about  the  same.  Therefore,  it  appears 
that  the  robot  is  capable  of  accurate  needle  placement. 

As  expected,  the  pain  score  post-treatment  was  significantly  less  than  the  pain  score 
pre-treatment  in  both  the  robot  and  manual  arms.  In  the  robot  arm,  pain  scores  fell 
from  a  mean  of  6.3  pre-treatment  to  1.8  post-treatment.  In  the  manual  arm,  pain  scores 
fell  from  6.0  pre-treatment  to  0.9  post-treatment.  Patients  had  to  sign  an  informed 
consent  form  and  were  generally  receptive  to  the  use  of  the  robot. 

4  Future  work 

We  are  in  the  process  of  submitting  our  progress  report  to  the  FDA  and  asking  for 
permission  to  enroll  additional  patients.  In  addition,  the  next  clinical  application  that 
we  are  investigating  is  lung  biopsy.  The  goal  is  to  use  the  robotic  system  to  assist  the 
physician  in  accurate  computed  tomography  (CT)  fluoroscopy-guided  needle  biopsy 
of  lung  nodules. 
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Purpose: 

This  paper  describes  an  algorithm  for  automatic  needle  placement  using  a  six  degree  of 
freedom  (DOF)  robot  during  minimally  invasive  spine  procedures.  This  automatic  needle 
placement  is  achieved  by  a  closed-loop  position  control  of  the  robot  end  effector 
provided  by  computer  vision.  The  term  fluoroscopy  servoing  is  given  to  this  process 
since  an  x-ray  fluoroscope  is  used  as  the  visualization  tool.  A  software  application  was 
developed  to  segment  the  needle  in  the  x-ray  images  to  find  its  position,  which  is  then 
used  as  feedback  to  control  the  robot  and  move  it  to  a  desired  target  point.  A  cadaver 
study  was  performed  to  test  the  accuracy  of  this  application.  Twenty  trials  were 
performed  and  the  robot  reached  the  desired  target  point  with  an  average  distance  error 
of  1.21mm. 

Methods: 

Fluoroscopy  servoing  is  a  relatively  new  concept  that  has  only  been  investigated  by  a 
few  other  researchers  in  the  literature.  Loser  and  Navab  discuss  a  method  of 
fluoroscopy  servoing  using  a  prototype  that  has  two  perpendicular  rotational  axes  which 
allow  3D  needle  rotation  around  a  fixed  point  [1].  They  present  an  automatic  image- 
guided  control  based  on  visual  servoing,  for  automatic  and  uncalibrated  needle 
placement  under  CT  fluoroscopy.  An  initial  evaluation  of  the  robot  showed  an  accuracy 
in  needle  placement  of  ±1. 6mm  in  a  small  pig  study.  Patriciu,  Stoianovici,  and 
colleagues  developed  a  targeting  method  based  on  a  portable  x-ray  fluoroscope  [2].  A 
pattern  matching  algorithm  running  on  a  video  acquisition  board  is  used  to  rapidly  locate 
a  ball  marker  that  was  placed  on  the  needle  in  the  x-ray  image.  They  conducted  a  set  of 
experiments  in  the  operating  room  using  a  C-arm  fluoroscope  and  a  phantom.  They 
used  a  2mm  ball  as  the  target  and  placed  it  80mm  below  the  skin  entry  point.  The 
maximum  error  over  25  experiments  was  less  than  1 .5mm. 

Our  goal  is  to  move  fluoroscopy  servoing  into  clinical  practice.  While  the  papers  noted 
above  described  the  fluoroscopy  servoing  concept  and  did  some  preliminary 
experiments,  cadaver  studies  were  not  done  in  the  interventional  environment.  In 
addition,  the  work  by  Patriciu  and  Stoianovici  required  a  ball  marker  on  the  end  of  the 
needle.  Our  work  builds  on  the  work  of  Loser  and  Navab  and  extends  it  to  include  a 
robust  needle  segmentation  capability  that  was  verified  in  the  cadaver  studies. 

At  Georgetown  University,  we  recently  completed  a  clinical  trial  of  20  patients  using  a  six 
degree-of-freedom  joystick  controlled  robot  for  needle  placement  for  spinal  nerve  blocks 
[3, 4].  One  of  the  first  patients  in  this  trial  is  shown  in  Figure  1  (all  figures  have  been  sent 
to  the  organizers  as  attachments  and  should  be  available  with  this  abstract.  In  case  this 
figure  is  not  available,  it  shows  the  interventional  radiologist  using  a  Joystick  controlled 
needle  driver  robot  for  spinal  nerve  blocks).  The  needle  placement  process  consists  of 
first  making  a  translational  move  to  place  the  needle  over  the  skin  entry  point,  then 
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aligning  the  needle  towards  the  nerve  root,  and  finally  driving  the  needle  to  the  nerve 
root,  all  under  X-ray  fluoroscopy.  We  intend  to  eventually  automate  these  steps.  As  a 
first  step,  in  this  paper  we  report  our  results  in  automating  the  translational  placement  of 
the  needle  at  a  selected  skin  entry  point.  Once  the  needie  is  at  the  skin  entry  point,  we 
plan  to  apply  the  algorithm  of  Loser  and  Navab  to  align  it  toward  the  target  in  future 
work. 

Our  software  application  was  written  using  C/C++  based  iibraries  and  toolkits.  The 
software  packages  employed  included; 

1)  Motion  control  library  application  programming  interface  for  the  robot  motion 
control  board  (MEI) 

2)  URobotics  library  from  Johns  Hopkins.  This  higher  level  library  is  built  on  the  MEI 
library  and  provides  additionai  functionality. 

3)  Matrox  imaging  library  (MIL)  which  provides  an  interface  to  the  Matrox  Meteor  II 
frame  grabber  card  used. 

4)  Fast  light  toolkit  (FLTK)  for  the  user  interface. 

5)  Visualization  toolkit  (VTK)  for  image  visualization  and  display. 

6)  Insight  segmentation  and  registration  tooikit  (ITK)  for  segmentation  of  the  needie. 

Figure  2  is  a  flow  diagram  of  the  procedure  used  for  tracking  the  needle  on  the 
fluoroscopic  image.  First,  the  image  is  acquired  using  the  digitai  frame  grabber  card. 
Using  this  initiai  image  as  input,  a  robust  but  relativeiy  slow  process  is  applied  for 
performing  an  initial  detection  of  the  needie.  The  output  of  this  process  is  a  iist  of  pixel 
coordinates  belonging  to  the  needle.  From  this  set  of  points,  a  number  of  heuristics  are 
applied  in  order  to  identify  the  tip  and  head  of  the  needle.  The  needle  head  should  be 
interpreted  as  being  the  end  of  the  needle  opposed  to  the  tip.  The  needle  tip  is  the  end 
that  is  clinically  relevant  here  since  it  must  be  positioned  at  the  entry  point  on  the 
patient’s  skin.  Once  the  initial  detection  is  completed  the  robot  will  move  and  the  needle 
will  be  detected.  This  is  done  until  enough  information  is  gathered  to  get  the 
transformation  from  image  to  robot  space  to  move  the  robot  to  the  desire  target  point. 
The  most  chalienging  aspect  of  tracking  the  needie  on  the  image  is  that  at  each  new 
position  the  needle  may  be  superimposed  on  different  anatomical  structures  that  alter 
the  needle’s  intensity  profile.  The  methods  used  provide  a  satisfactory  degree  of 
robustness  against  the  cases  of  superposition  commonly  observed  in  typical  clinical 
images.  Figure  3  shows  the  flow  diagram  of  the  entire  process  used  for  detecting  the 
needle  in  the  fluoroscopic  images  during  the  calibration  process. 

To  use  the  information  extracted  from  the  fluoroscopic  image  to  guide  the  translational 
movements  of  the  robot,  we  first  need  to  determine  the  transform  that  maps  the 
coordinate  system  of  the  robot  into  the  image  coordinate  system.  The  direct 
transformation  from  one  planar  coordinate  system  to  the  other  is  given  by  a  two 
dimensionai  affine  transform. 

To  get  the  input  for  the  affine  transform,  a  caiibration  procedure  needs  to  be  done.  This 
procedure  establishes  the  relationship  between  the  camera  frame  and  the  robot  frame. 
The  robot  is  moved  to  3  known  positions  and  the  coordinates  in  both  robot  space  and 
image  space  are  recorded.  Image  segmentation  is  used  to  find  the  position  of  the 
needle  in  the  image.  These  3  point  coordinates  are  used  to  compute  the  two 
dimensional  transformation  between  the  two  coordinate  systems. 

Results: 
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To  test  the  robustness  of  the  software  application  two  experiments  were  completed.  The 
first  experiment  consisted  of  alignment  of  the  needle  under  joystick  control  of  a  human 
operator.  The  second  was  done  using  the  fluoroscopy  servoing  algorithm  for  automatic 
needle  alignment.  Both  of  these  experiments  were  done  in  the  interventional  suite  at 
Georgetown  University  using  a  cadaver.  This  was  done  to  compare  the  speed  and 
accuracy  of  the  manual  process  with  the  automatic  process.  The  experimental  set-up  is 
shown  in  Figure  4. 

For  the  manual  needle  alignment  a  2  mm  metal  BB  marker  was  placed  on  the  back  of 
the  cadaver  and  used  as  the  target.  Under  continuous  x-ray  fluoroscopy  the  robot  was 
moved  by  a  human  operator  using  the  Joystick  until  the  needle  was  placed  over  the  BB. 

For  the  second  experiment  an  artificial  target  was  selected  by  moving  the  robot  to  an 
arbitrary  position  (to  sirriulate  a  desired  skin  entry  point).  Once  the  robot  was  there  the 
needle  tip  was  selected  as  the  target.  At  this  point,  the  robot's  position  was  recorded. 
The  robot  was  then  moved  to  its  origin  (0,  0),  and  the  calibration  process  was  done. 
When  the  calibration  process  finished,  the  fluoroscopy  servoing  algorithm  was  used  to 
move  the  robot  to  the  target  position.  The  robot  coordinates  were  then  recorded  at  this 
position. 

Once  the  20  trials  were  done  the  x-ray  images  were  saved  in  DICOM  format  and 
converted  to  TIFF  images  using  a  public  domain  software  package.  A  typical  A/P  x-ray 
image  is  shown  in  Figure  5.  Then,  the  measuring  tool  in  Adobe  Photoshop  5.5  was  used 
to  compute  the  distance  from  the  needle  tip  to  the  center  of  the  BB.  The  distance  from 
the  target  to  the  tip  of  the  needle  was  measured  in  the  x  and  y  directions  and  a  root 
mean  square  distance  was  computed. 

The  difference  in  errors  between  the  manual  trials  and  the  automatic  trials  was  very 
small.  On  average  the  measured  root  mean  square  error  for  the  manual  trials  was  1.39 
+/-  1.02mm  and  1.21  +/-0.66mm  for  the  automatic  trials.  Both  of  these  are  clinically 
acceptable. 

Conclusion; 

This  paper  described  the  use  of  fluoroscopy  servoing  for  automatic  needle  alignment 
using  a  medical  robot.  A  software  application  was  developed  to  frame  grab  the  X-ray 
images,  segment  the  needle,  and  automatically  move  the  needle  tip  to  a  physician 
selected  skin  entry  point.  A  cadaver  study  was  completed  in  the  interventional  suite 
which  showed  the  accuracy  of  automatic  placement  was  similar  to  manual  placement. 

This  is  a  first  step  in  incorporating  fluoroscopy  servoing  for  automatic  needle  placement 
in  clinical  procedures.  We  intend  to  continue  this  work  by  adding  an  orientational 
capability  and  doing  further  validation  studies  before  moving  to  a  clinical  trial. 
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The  pxupose  of  this  study  is  to  report  and  analyze  patient  skin  motion  data  collected 
during  CyberKnife  stereotactic  radiosurgery.  The  CyberKnife  is  a  radiation  treatment 
system  that  incorporates  a  robotic  arm  to  precisely  position  a  linear  accelerator.  While 
this  capability  is  not  currently  commercially  available,  the  CyberKnife  could  be 
programmed  to  move  the  radiation  beam  in  real-time  to  compensate  for  organ  motion. 
This  study  serves  as  a  first  step  towards  our  long-term  goal  of  using  skin  motion  to 
predict  internal  organ  motion.  This  may  lead  to  more  precise  radiation  treatment  delivery 
for  mobile  target  volumes.  To  this  date  we  have  collected  and  analyzed  skin  motion  data 
on  four  patients  (two  sacrum,  one  lung,  one  pancreas).  The  movement  amplitudes  of 
individual  skin  markers  ranged  from  3.1  mm  to  14.8  mm  with  a  median  of  7.5  mm. 

1.  INTRODUCTION 

1.1  Stereotactic  Radiosurgery 

Stereotactic  Radiosurgery  (SRS)  was  developed  in  the  early  1980s  with  the  objective  of 
giving  a  high,  conformal,  single  treatment  dose  to  a  lesion  while  sparing  healthy  tissue. 
The  first  SRS  treatments  were  done  with  conventional  external  beam  machines  using 
multiple  treatment  fields  and  tertiary  circular  collimators  located  close  to  the  patient.  The 
GammaKnife  used  the  same  concept  of  multiple  beams,  but  instead  of  a  linac,  the  beams 
were  generated  by  collimated  ^Co  sources.  Both  methods,  external  x-ray  beam  and 
GammaKnife  SRS,  used  forward  planning  to  calculate  target  doses. 

The  arrival  of  multileaf  collimators  (MLCs)  made  it  possible  to  change  field  shapes 
quickly  and  match  the  projection  of  the  tumor  shape.  In  intensity  modulated  radiation 
frierapy  (IMRT),  this  technical  advancement  is  used  to  target  and  differentially  dose  sub¬ 
volumes  of  the  lesion  from  multiple  directions,  which  adds  up  to  a  conformal  dose  for  the 
whole  target  and  better  avoidance  of  critical  organs  [1].  The  CyberKnife  (Accuray  Inc., 
Sunnyvale,  CA),  while  technologically  very  different  from  IMRT,  employs  a  similar 
mathematical  concept  for  dose  optimization.  Out  of  a  library  of  about  1400  beams,  the 
treatment  planning  system  chooses  the  beams  with  the  best  target  coverage  and  critical 
structure  avoidance,  weighing  them  appropriately. 

The  dose  constraints  for  IMRT  and  CyberKnife  are  too  complex  to  be  handled  by 
forward  planning  methods.  Therefore,  inverse  planning  algorithms  were  developed 
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starting  in  the  late  1980s.  The  desired  3D  dose  image  constraints  are  put  in,  which  the 
algorithm  mathematically  inverses  to  calculate  the  ou^ut  dose.  Because  the  exact 
solution  would  include  negative  and  imaginary  intensities,  in  practice  iterative  search 
algorithms  are  used.  So-called  cost  functions  are  used  in  most  inverse  planning 
algorithms  to  measure  the  difference  between  the  calculated  and  the  prescribed  dose. 

1.2  CyberKnife  and  Respiratory  Motion 


The  CyberKnife  [2,3]  consists  of  a 
miniature  6  MeV  linac  mounted  on  a 
robot.  The  robot  arm  has  a  full  six 
degrees  of  freedom.  Two  orthogonal 
x-ray  cameras  with  corresponding 
amorphous  silicon  detectors  allow 
verification  of  the  patient’s  position 
before  each  treatment  beam.  The 
system  automatically  corrects  for 
translations  up  to  10  mm.  Rotational 
movements  are  displayed  on  the 
monitor  and  can  be  corrected  by 
rotating  the  treatment  couch,  if 
necessary.  Figure  1  shows  the 
CyberKnife  Suite  in  the  Georgetown 
University  Hospital  Department  of 
Radiation  Medicine. 

Radiation  treatment  of  moving  lesions 
while  simultaneously  sparing  non- 
cancerous  tissue  has  been  a  long¬ 
standing  challenge  in  radiation 
therapy.  The  main  focus  in  external 
beam  radiotherapy  has  been  research 
on  breath-holding  and  respiratory 
gating  techmques.  While  both  methods  have  improved  radiotherapy  treatment,  certain 
issues  as  prolonged  treatment  time  and  patient  position  verification  have  largely  remain 
unresolved.  The  CyberKnife  system  provides  the  unique  possibility  to  do  real-time 
patient  and  tumor  motion  tracing  during  stereotactic  radiosurgery.  If  the  respiratory 
motion  of  a  lesion  could  be  tracked  in  real-time,  the  lesion  position  could  be 
communicated  via  a  feedback  loop  to  the  robot,  which  could  correct  for  the  motion. 
Because  the  high  dose  rate  for  the  required  treatment  time  makes  it  impractical  to  use 
fluoroscopic  tracking,  some  researchers  have  used  skin  markers  for  external  tracking  and 
developed  a  correlation  model  to  the  internal  fiducial  markers.  Comparing  the  predicted 
internal  marker  position  with  the  position  calculated  by  the  x-ray  cameras  can  be  used  for 
correlation  model  verification.  Preliminary  studies  [4,5]  using  a  linear  correlation  model 
have  been  done.  Our  study  serves  as  a  first  step  to  verify  those  results  and  possibly 
improve  respiratory  motion  tracking  algorithms. 


Figure  1 :  CyberKnife  Suite  at  Georgetown 
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2.  METHODS 

The  authors  received  institutional  review  board  (IRB)  approval  for  a  study  to  track  skin 
motion  during  regularly  scheduled  CyberKnife  treatments.  The  skin  motion  tracking  was 
done  using  a  commercial  infrared  tracking  system  (Polaris,  Northern  Digital,  Ontario, 
Canada).  The  Polaris  can  track  the  motion  of  non-invasive  hemispherical  sensors,  which 
were  temporarily  attached  to  the  patients’  gown  or  clothing  using  double-sided  tape.  The 
non-invasive  sensor  is  shown  in  Figure  2  and  the  optical  tracking  system  is  shown  in 
Figure  3.  The  infrared  tracking  system  emits  infrared  light  that  is  reflected  by  these 
hemispheres.  The  reflections  are  detected  by  the  infrared  tracking  system  and  used  to 
determine  the  position  of  the  sensors.  A  mock  treatment  session  is  shown  in  Figure  4. 


Figure  2:  Non-invasive  sensor  ^  t  x-  j 

compared  to  a  dime  (three  or  four  f'f"®  <«ckmg  system 

sensors  were  placed  on  the  pattern)  ■” 


Figure  4:  Mock  treatment  setup 
CyberKnife  system  and  saved  in  log  files 


When  the  patients  were  being  prepared 
for  treatment,  the  Polaris  was  set  up  in  the 
treatment  room  near  the  foot  of  the  table. 
The  Polaris  was  controlled  from  a  PC 
located  outside  the  treatment  room.  The 
long  distance  between  the  Polaris  and  the 
PC  (approximately  75  feet)  required  the 
use  of  the  RS-422  communications 
standard.  An  adapter  and  special  cable 
was  required  to  convert  the  signal  from  its 
native  RS-232  format.  A  custom  software 
program  was  written  to  follow  the 
position  of  the  skin  markers  and  save  this 
data  to  disk. 

Separately,  the  positions  of  the  internal 
fiducials  were  recorded  by  the 
.  These  internal  fiducials  are  implanted  before 


treatment  so  the  CyberKnife  can  follow  their  motion  using  the  x-ray  cameras  and  flat 
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panel  detectors.  The  internal  fiducial  coordinates  were  extracted  from  the  log  file  using  a 
Perl  script  and  converted  to  a  Matlab  compatible  file  format.  The  skin  marker  coordinates 
were  also  imported  into  Matlab.  The  data  was  then  analyzed  using  Matlab  scripts. 

3.  RESULTS 

Four  patients  have  been  enrolled  in  the  study  to  date  and  the  results  are  given  in  Table  1. 
Two  of  the  patients  were  treated  in  three  firactions  (designated  by  Fx  in  the  table).  Four 
markers  were  placed  on  the  first  three  patients,  and  three  markers  on  the  fourth  patient. 
Movement  amplitudes  of  individual  markers  ranged  from  3.1  mm  to  14.8  mm  with  a 
median  of  7.5  mm;  the  centroid  amplitudes  were  between  2.1  mm  and  6.0  mm.  The 
average  movement  of  the  skin  marker  centroid  was  4.7  mm,  3.3  mm,  2.6  mm  and  6.0 
mm. 


Table  1 :  Skin  marker  motion  for  four  patients 


Fx 

#1  (mm) 

#2  (mm) 

#3  (mm) 

#4  (mm) 

Centroid  (mm) 

Patient  1 

1 

6.9 

7.6 

7.7 

N/A 

4.7 

Patient  2 

1 

4.1 

6.6 

5.9 

4.4 

3.7 

2 

8.0 

12.2 

10.9 

6.2 

3.4 

3 

8.1 

12.0 

11.6 

7.5 

2.9 

Patient  3 

1 

6.3 

7.7 

8.5 

5.6 

2.1 

2 

6.5 

5.8 

7.0 

4.2 

2.3 

3 

14.8 

14.2 

12.9 

13.5 

3.3 

Patient  4 

1 

9.9 

5.1 

3.1 

N/A 

6.0 

The  centroids  of  the  internal  fiducial  markers  were  extracted  fi'om  the  CyberKnife  log 
files.  The  centroids  moved  2.5  mm  and  1 .7  mm  for  the  two  patients  with  fiducials  placed 
in  the  sacrum.  The  fiducial  markers  in  the  pancreas  moved  3.5  mm,  and  2.0  mm  in  the 
lung.  There  were  no  statistical  differences  in  either  external  or  internal  marker 
movements  for  the  first,  second  and  third  part  of  one  treatment. 

The  skin  motion  data  was  then  analyzed  using  the  Fast  Fourier  Transformation  from 
Matlab.  Breathing  patterns  for  a  particular  patient  remain  consistent  over  several 
treatment  stages.  Breathing  patterns  vary  widely  between  patients;  some  individuals  have 
a  very  consistent  and  predictable  breathing,  others  show  irregular  breathing  patterns.  In 
general,  breathing  frequencies  ranged  from  13  Hz  to  23  Hz. 

Figure  5  shows  the  motion  of  the  skin  marker  centroid  of  Patient  4  of  our  sample  with  the 
corresponding  FFT  analysis.  The  patient  was  diagnosed  with  a  bronchoalveolar 
carcinoma  of  the  limg  and  underwent  multiple  thoracic  surgeries  including  upper 
lobectomy,  right  middle  lobectomy,  a  left  upper  lobectomy,  and  right  lower  wedge 
resection.  She  was  wearing  nasal  oxygen  and  experienced  some  anxiety  during  the 
treatment.  While  her  breathing  cycle  is  fairly  regular,  two  distinct  features  could  be  seen 
from  Figure  5.  The  first  feature  was  the  very  high  breathing  rate  at  about  21  breaths  per 
minute.  Second,  every  few  breaths  she  took  a  longer  inhalation,  which  also  shows  two 
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local  maxima.  Three  examples  of  this  can  be  seen  in  Figure  5a  at  38  s,  68  s  and  78  s  after 
the  start  of  the  recording. 


Figure  5:  motion  of  skin  marker  centroid  of  patient  4  (left)  and  FFT  analysis  (right) 

It  was  foimd  that  great  care  needed  to  be  taken  in  placing  the  skin  markers  and  in 
arranging  the  environment.  For  example,  items  that  reflect  light  such  as  belt  buckles  or 
shirt  buttons  can  cause  crosstalk,  i.e.,  five  instead  of  four  skin  markers  were  recognized. 
Similar  problems  would  occur  if  skin  markers  were  placed  too  close  together.  Therefore, 
patients  were  asked  to  change  into  a  hospital  gown  for  their  treatment,  and  the  markers 
were  placed  at  least  10  cm  apart.  We  also  chose  to  do  a  software  cut  on  markers  more 
than  one  meter  away  from  the  expected  direction.  The  marker  placement  also  was  varied 
according  to  the  patient’s  position  on  the  treatment  table  (head  first  supine  or  feet  first 
supine).  For  feet  first  supine  patients,  the  markers  were  placed  more  inferiorly  on  the 
abdomen. 

4.  DISCUSSION 

Our  initial  results  showed  that  skin  motion  tracking  with  the  Polaris  in  the  CyberKnife 
treatment  room  linked  to  a  PC  outside  the  room  is  technically  feasible.  There  was  no 
interference  of  the  high  frequency  of  the  CyberKnife  linac  with  the  Polaris  or  the  signal 
transfer.  A  position  for  the  Polaris  could  be  found  so  that  it  is  not  in  the  way  of  the 
robotic  arm  and  close  enough  to  the  patient  to  ensure  stable  tracking.  For  every  patient 
studied,  the  amplitude  of  the  skin  motion  was  significantly  larger  than  the  tracking  error 
of  the  Polaris  (manufacturer’s  accuracy  is  0.35  mm).  Placing  only  three  markers  on 
smaller  patients  could  minimize  crosstalk  between  skin  markers.  There  are  still  some 
phantom  fiducials  in  the  tracking.  We  assume  their  source  is  the  reflection  of  infrared 
light  from  some  not  yet  known  reflecting  surface  in  the  treatment  room.  Most  of  the 
phantom  markers  could  be  eliminated  by  applying  a  software  cut  on  all  markers  further 
than  one  meter  from  the  initid  marker  placement. 

We  fmmd  that  internal  fiducial  motion  for  the  patients  treated  so  far  was  minimal.  This  is 
in  part  related  to  the  patient  selection  process  for  CyberKnife  treatment.  Currently,  our 
patient  population  is  about  50%  intracranial  cases,  40%  spine  cases  and  10%  non-CNS 
cases  (paraspinal,  pancreas  and  lung  lesions  attached  to  the  mediastinum). 
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The  CyberKnife  stereotactic  radiosurgery  system  offers  new  possibilities  for  actively 
compensating  for  respiratory  motion  during  radiation  treatment.  To  implement  this 
compensation,  methods  need  to  be  developed  for  tracking  organ  movement.  The 
correlation  of  skin  markers  to  internal  fiducial  markers  is  a  promising  tool.  The 
complexity  of  breathing  will  require  advanced  methods  of  modeling  and  signal 
processing  techniques.  There  is  also  a  need  to  gather  breathing  data  across  a  spectrum  of 
patients,  as  this  data  is  not  readily  available  in  the  literature. 

The  data  presented  in  this  paper  is  intended  to  be  a  first  step  in  these  directions.  We  also 
intend  to  build  on  the  work  of  the  CyberKnife  group  at  Stanford  and  their  reports  on  skin 
motion  and  internal  organ  motion.  The  preliminary  results  shown  here  indicate  that 
breathing  patterns  are  complex,  but  that  skin  motion  may  be  characterized  by  the  centroid 
of  the  markers  as  the  movement  of  the  centroid  was  slight. 
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Abstract 


Introduction:  With  the  introduction  of  Synchrony,  the  need  for  a  programmable 
respiratory  motion  simulator  (RMS)  becomes  apparent.  The  RMS  can  serve  the  multiple 
purposes  of  being  used  for  dose  delivery  verification  for  Synchrony  treatments  and  as  a 
simulator  to  enable  future  technical  development. 

Methods:  Two  independent  3D  motion  platforms  for  skin  (SMS)  and  tumor  (TMS)  were 
designed  and  constructed.  The  technical  requirements  for  motion  amplitude  and  speed  of 
the  respective  platforms  were  based  on  patient  data  taken  at  Georgetown  University 
Hospital  (SMS)  and  published  data  (TMS).  The  geometry  of  the  CyberKnife  system  was 
considered  to  ensure  collision  avoidance  and  avoid  metal  parts  in  the  imaging  area. 

Results:  The  RMS  was  built  completely  fi'om  off-the-shelf  components.  Wiring, 
assembly  and  software  development  were  done  in-house  and  the  initial  tests  have  been 
completed  in  the  CyberKnife  Suite.  Respiratory  data  taken  from  a  Irmg  patient  treated  on 
CyberKnife  could  be  accurately  reproduced  using  the  simulator.  Skin  and  liver  motion 
data  fi-om  an  independent  pig  study  could  also  be  simulated.  A  film  cube  was  treated  on 
the  RMS  with  and  without  respiratory  motion.  A  significant  difference  in  dose 
distribution  for  the  static  vs.  moving  (non-compensated)  regular  CyberKnife  treatment 
was  demonstrated. 

Conclusions:  The  RMS  can  reproduce  skin  and  tumor  motion  with  high  precision.  The 
simulator  will  be  used  in  the  CyberKnife  suite  for  phantom  treatments,  dose  verification, 
and  future  technical  developments. 
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1.  Introduction 

The  CyberKnife  has  been  used  successfully  for  extracranial  stereotactic  radiosurgery  [1- 
5]  for  tumors  which  do  not  show  significant  respiratory  motion.  The  development  of 
computer  technology,  together  with  the  6D  motion  characteristics  of  a  robot  moimted 
linear  accelerator  and  advanced  medical  imaging,  makes  it  technically  possible  to  follow 
respiratory  motion  in  real-time.  An  alternative  solution,  moving  the  leaves  of  a  mini¬ 
multileaf  collimator  on  external  beam  machines,  could  potentially  achieve  the  same 
purpose  for  tumor  motions  perpendiciilar  to  the  beams-eye  view.. 

The  challenge  for  both  approaches  lies  in  solving  the  problem  of  tracking  the  quasi- 
periodic  motion  of  an  internal  tumor  with  high  precision.  Continuous  fluoroscopy,  which 
can  be  used  in  relatively  short  procedures  as  angiography,  is  not  feasible  for  the  time 
period  required  for  a  CyberKnife  treatment.  If  a  correlation  between  internal  tumor 
motion  and  skin  motion  can  be  reliably  established,  the  skin  motion  could  be  used  to 
predict  the  position  of  the  tumor.  The  correlation  must  be  flexible  enough  to  accormt  for 
changes  in  respiratory  amplitude  and  frequency  during  the  treatment  [6].  Typically,  the 
respiratory  rate  slows  down  during  the  first  few  minutes  of  the  patient  lying  on  the 
treatment  couch.  Falling  asleep,  or  waking  up  during  the  treatment,  will  change  breathing 
patterns.  Even  if  the  system  is  “trained”  to  recognize  and  follow  a  specific  patient’s 
pattern  during  a  fraction,  inter-fractional  changes  may  occur.  If  the  patient  experiences 
pain  relief  during  the  course  of  the  treatment,  intra-fraction  breathing  will  stay 
unchanged,  but  inter-fractional  breathing  patterns  may  change.  Safety  requires  that  the 
system  recognizes  atypical  patterns,  such  as  a  cough  or  a  yawn,  and  stop  the  radiation 
treatment  vuitil  a  regular  pattern  is  reestablished. 


2.  Methods  and  Materials 

We  have  studied  breathing  patterns  of  patients  during  CyberKnife  treatment  using  the 
Polaris  infrared  tracking  system  (Northern  Digital,  Ontario,  Canada)  under  an  IRB 
approved  protocol.  Because  patients  are  selected  with  tumors  which  do  not  move  much  or 
at  all  with  respiratory  motion,  there  was  no  correlation  to  internal  fiducial  markers  to  be 
established.  However,  we  were  able  to  study  respiratory  motion  amplitudes,  skin  marker 
deformation,  intra-  and  inter-fractional  changes  of  breathing  patterns  for  a  wide  variety  of 
patients  [7].  The  next  step  is  to  develop  algorithms  to  predict  the  direction  of  skin  marker 
motion  250  ms  ahead,  which  corresponds  to  the  CyberKnife  system  response  time  for  a 
Synchrony  treatment.  To  test  the  algorithms,  we  plan  to  use  of  respiratory  motion  data 
taken  during  animal  studies.  Therefore,  we  decided  to  build  a  respiratory  motion 
simulator  (RMS). 

A  respiratory  motion  simulator  for  the  CyberKnife  should  have  two  independent  stages 
capable  of  3D  movement  for  internal  and  skin  movements.  Both  stages,  the  tumor  motion 
simulator  (TMS)  and  the  skin  motion  simulator  (SMS),  should  be  programmable  with 
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real  patient  motion  data  taken  during  CyberKnife  treatments.  Because  a  RMS  could  be 
used  as  a  dose  verification  phantom  for  Synchrony  treatments,  the  TMS  should  be  able  to 
carry  the  weight  of  a  film  cube  enclosed  in  a  dose  buildup  material.  No  metal  should  be 
in  the  imaging  region  to  allow  for  fiducial  tracking.  The  overall  size  is  limited  by  the 
treatment  table,  safety  box  (robot/RMS  collision  avoidance)  and  no-metal  zone.  The 
relative  position  accuracy  should  be  better  than  0.2  mm.  The  maximum  breathing 
amplitudes  are  3  cm  in  all  directions  and  the  maximum  breathing  frequency  is  30  breaths 
per  minute.  The  external  load  for  the  TMS  is  5  kg;  the  SMS  does  not  carry  load.  The 
weight  of  each  platform  is  approximately  12  kg. 


Figure  1:  CAD  model  of  RMS 


Figure  2:  RMS  in  Cyberknife  Suite 


The  SMS  uses  three  linear  slides 
(Velmex)  to  provide  motion  on 
three  axes.  The  TMS  uses  two 
linear  slides  for  the  first  two  axes, 
and  a  linear  actuator  for  the  third 
axis.  Each  axis  is  driven  by  a 
motor,  optical  encoder,  and 
gearhead  assembly  (Maxon).  The 
combination  of  motor,  optical 
encoder  and  gearhead  is  accurate  to 
1/9600  revolutions  of  the  motor, 
which  corresponds  to  a  resolution 
of  0.001mm  on  the  sliders.  The 
motion  simulator  is  controlled  by  a 
six  axes  standalone  Motion  Control 
Card  (Galil)  that  can  be  commanded 
from  outside  the  treatment  room  via 
Ethernet. 


Time  (s) 


1 


Frequency  (Hz)  at  Hp 


Figure  3:  Skin  marker  on  base  and  tip  of  the  SMS  stage 
were  moved  in  a  sinusoidal  motion.  A  frequency  analysis 
was  done  to  check  for  vibrations. 
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3.  Results 

The  RMS  was  tested  in  the  CyberKnife  Suite.  First,  we  separately  tested  both  stages  with 
a  3D  sinusoidal  motion.  The  motion  programmed  motion  frequency  was  0.2  Hz.  Two 
fiducial  markers  were  used  on  each  stage:  one  on  the  bases  of  the  motion,  and  the  second 
one  on  the  phantom  platform/tip  of  the  beam.  The  data  was  analyzed  for  vibration, 
position  accuracy  and  timing.  Figure  3  shows  the  results  for  the  SMS  stage. 

These  tests  were  followed  by  a  16  minute  breathing  motion  reproduction  of  skin 
respiratory  motion  data  from  a  CyberKnife  patient  treatment  for  both  stages.  The  data 
used  was  the  centroid  of  four  fiducials  placed  on  this  patient’s  abdomen. 


TMS  Breathing  Analysis 


The  RMS  was  then  run  with  skin  and  liver  motion 
data  from  a  pig  study.  Both  stages  could  be  run, 
and  no  collision  problems  between  the  two  stages 
occurred.  Several  x-ray  images  of  a  film  cube 
placed  on  the  TMS  platform  were  taken  with  the 
CyberKnife  image  system.  No  metal  showed  in 
the  vicinity  of  the  fiducials  within  the  film  block. 
Parts  of  the  motor  of  the  lowest  TMS  linear  slider 
showed  at  the  periphery  of  the  Camera  A  image, 
which  will  not  interfere  with  fiducial  tracking 
within  the  range  of  TMS  motion. 
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To  demonstrate  the  distortions  of  dose 
distributions  in  a  tumor  for  uncorrected 
breathing  motion,  we  did  two  phantom 
treatments.  The  film  cube  was  loaded  with 
two  sheets  of  GaFchromic  film  placed 
perpendicularly  to  each  other.  A  CT  of  the 
film  cube  on  the  TMS  platform  was  taken 
and  transferred  to  the  CyberKnife.  A 

spherical  tumor  was  contoured  and 
planned  with  a  dose  of  3000  cGy  to  the 
80%  isodose  line.  Three  fractions  of  this 
treatment  were  delivered.  For  the  first 
fraction,  the  film  cube  was  aligned  and 
treated  with  no  motion  at  all.  Imaging  was 
disabled;  therefore  no  motion 
compensation  was  done.  For  the  second 
fraction  we  moved  the  TMS  in  a  sinus 
motion  with  5  mm  amplitude  in  the 
anterior-posterior  and  left-right  direction 
and  10  mm  in  the  inferior-superior 
direction.  The  phantom  was  imaged  before 
every  treatment  beam  and  CyberKnife 
then  corrected  for  translational  shifts. 

4.  Conclusion 

The  RMS  fits  the  physical  space  requirements  and  we  have  completed  initial  testing  in 
the  CyberKnife  Suite.  Both  stages  can  be  independently  programmed  to  reproduce  patient 
and  pig  data.  No  metal  parts  interfered  with  fiducial  tracking  of  a  phantom  on  the  TMS 
platform.  The  phantom  could  be  run  from  the  CyberKnife  treatment  console  during  beam 
delivery.  The  basic  hardware  design  fulfilled  our  expectation.  Small  changes,  e.g.  a  less 
vibrating  SMS  beam  and  further  improvement  of  the  software,  will  be  done.  The  RMS 
can  reproduce  skin  and  tumor  motion  with  high  precision.  The  simulator  will  be  used  in 
the  CyberKnife  suite  for  phantom  treatments,  dose  verification  and  future  technical 
developments. 
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ABSTRACT 

This  paper  describes  a  computer  program  to  utilize  electromagnetic  tracking  guidance  during  insertion  of  peripherally 
inserted  central  catheters.  Placement  of  a  Peripherally  Inserted  Central  Catheter  (PICC)  line  is  a  relatively  simple, 
routine  procedure  in  which  a  catheter  is  inserted  into  the  veins  of  the  lower  arm  and  threaded  up  the  arm  to  the  vena  cava 
to  sit  just  above  the  heart.  However,  the  procedure  requires  x-ray  verification  of  the  catheter  position  and  is  usually  done 
under  continuous  fluoroscopic  guidance.  The  computer  program  is  designed  to  replace  fluoroscopic  guidance  in  this 
procedure  and  make  PICC  line  placement  a  bedside  procedure.  This  would  greatly  reduce  the  time  and  resources 
dedicated  to  this  procedure.  The  physician  first  goes  through  a  quick  registration  procedure  to  register  the  patient  space 
with  the  computer  screen  coordinates.  Once  registration  is  completed,  the  program  provides  a  continuous,  real-time 
display  of  the  position  of  the  catheter  tip  overlaid  on  an  x-ray  image  of  the  patient  on  an  adjacent  computer  screen.  Both 
the  position  and  orientation  of  the  catheter  tip  is  shown.  The  display  is  very  similar  to  that  shown  when  using 
fluoroscopy. 

Keywords:  electromagnetic  tracking,  PICC,  catheter  insertion 

1.  INTRODUCTION 

The  peripherally  inserted  central  catheter  line  is  often  used  in  patients  requiring  long  term  or  home  intravenous  access 
for  a  wide  variety  of  medical  conditions.  It  is  inserted  in  the  lower  arm,  feeds  up  the  subclavian  vein,  then  the 
brachiocephalic  vein,  and  finally  it  enters  the  superior  vena  cava.  It  does  not  become  as  easily  infected  or  dislodged  as  a 
shorter  intravenous  line  and  is  safer  to  insert  and  take  care  of  than  a  central  venous  catheter  (CVC)  line.  Although  the 
PICC  line  is  a  long-term  use  catheter,  it  must  be  changed  if  it  becomes  infected  or  remains  in  the  patient  too  long.  The 
procedure  is  carried  out  fairly  often  for  a  variety  of  treatments.  For  these  reasons,  PICC  line  insertion  is  a  procedure 
done  many  times  a  day  in  every  hospital,  and  requires  a  fair  amount  of  hospital  resources  and  time. 

Electromagnetic  guidance  is  a  modality  that  may  be  useful  for  guiding  minimally  invasive  surgical  procedures.  It  can 
provide  accurate  tracking  without  the  line-of  sight  requirements  of  optical  trackers.  This  characteristic  makes  it  possible 
to  use  electromagnetic  tracking  systems  for  tracking  surgical  implements  inside  the  body  without  an  open  incision.  For 
instance,  electromagnetic  guidance  of  catheters  is  currently  used  in  minimally  invasive  cardiac  surgery  [1].  Our  research 
group  is  investigating  using  electromagnetic  tracking  for  applications  as  diverse  as  needle  placement,  RF  ablation,  and 
organ  motion  tracking  [1][2][4][5]. 
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For  our  system,  we  have  chosen  to  use  the  AURORA  Electromagnetic  Tracker 
(Northern  Digital,  Inc,  Waterloo,  Canada,  Figure  1).  The  AURORA  consists  of  a 
field  generator  (right  hand  side  of  Figure  1),  a  control  unit  (left  hand  side  of 
Figure  1),  and  small  coil  sensors  that  can  be  embedded  in  catheters,  needles,  and 
other  instruments.  The  coils  return  the  sensor  position  and  orientation  with  five 
degrees  of  freedom  (x,  y,  z  translations  and  two  orientations).  It  has  been 
demonstrated  to  give  reasonably  good  accuracy  [6],  is  small,  reasonably  priced, 
and  easily  portable.  Using  this  method  of  guidance  could  greatly  cut  down  on  the 
number  of  PICC  line  cases  that  would  need  access  to  fluoroscopy  equipment. 


Figure  1 :  AURORA  control  unit 
_ and  field  generator _ 


2.  METHODS 


2.1  PICC  Placement 

The  procedure  for  inserting  a  PICC  line  is  very  simple.  The  physician  needs  to  see  that  the  catheter  goes  up  the 
subclavian  vein,  then  the  brachiocephalic  vein,  and  ends  up  in  the  superior  vena  cava.  There  are  basically  two  possible 
problems  with  the  procedure  which  would  have  to  be  easily  recognizable  by  our  system.  The  physician  must  be  able  to 
recognize  if  the  catheter  takes  a  path  into  a  wrong  vein.  For  instance,  the  catheter  may  go  up  into  the  jugular  vein  instead 
of  the  vena  cava,  or  miss  the  vena  cava  and  pass  across  to  the  other  side  of  the  body.  The  physician  must  also  be  able  to 
see  if  the  catheter  becomes  stuck  in  a  vein  or  doubles  back  on  itself 

Fluoroscopic  guidance  gives  a  very  good  view  of  exactly  where  the  catheter  is  in  the  patient’s  anatomy  at  any  time. 
However,  it  exposes  the  patient  to  some  amount  of  radiation.  Our  goal  was  to  use  electromagnetic  tracking  to  give  the 
physician  a  view  similar  to  what  he  would  see  with  digital  fluoroscopy,  but  without  the  radiation.  We  consulted  with  an 
interventional  radiologist  experienced  in  the  procedure,  and  were  given  the  following  requirements  for  our  system.  Our 
program  must  run  in  real  time  and  update  often  enough  to  give  the  physician  continuous  information  about  the  location 
of  the  catheter.  The  display  must  show  the  path  and  location  of  the  catheter  clearly  enough  to  detect  its  taking  a  wrong 
path  or  getting  stuck  in  a  vein.  In  addition  to  this,  the  system  is  portable  and  able  to  be  run  off  a  laptop  computer  and  can 
be  moved  into  any  patient’s  room. 

2.2  Software  Design 

Since  this  project  was  begun  by  students  as  part  of  a  class  at  Johns  Hopkins  University,  our  program  relies  on  the  open 
source  Computer  Integrated  Surgery  C-h-  library  developed  by  the  Engineering  Research  Center  at  Johns  Hopkins.  This 
software  is  used  for  the  AURORA  drivers  as  well  as  the  code  for  mathematical  operations  such  as  calculation  of  frame 
transformations.  The  software  is  also  dependent  on  the  Fast  Light  Toolkit  1.10  (FLTK)  for  the  graphical  display  and  user 
interface  control. 

The  first  step  in  the  procedure  for  using  our  software  is  to  perform  a  pivot  calibration  of  the  catheter.  In  the 
electromagnetically  tracked  catheters  that  were  used  for  this  project,  the  sensor  coil  was  not  necessarily  in  the  tip  of  the 
catheter,  but  may  be  offset  by  as  much  as  1-2  mm.  The  physician  is  asked  to  take  6  readings  with  the  catheter  tip 
touching  a  single  point  at  different  orientations.  From  this,  the  average  distance  is  calculated  from  the  coil  to  the  physical 
tip  of  the  catheter.  Data  from  the  pivot  calibration  can  be  saved  and  reloaded  if  the  physician  would  like  to  reuse  the 
catheter. 

In  the  interest  of  keeping  the  format  of  the  display  similar  to  the  fluoroscopy  display  that  the  physician  is  accustomed  to, 
the  catheter  location  is  overlaid  on  a  chest  x-ray  image  of  the  patient.  Patients  that  require  a  PICC  line  often  have  other 
medical  problems  and  in  the  past,  may  have  needed  a  chest  x-ray.  The  program  allows  the  physician  to  load  either  a 
patient-specific  x-ray  or  a  generic  x-ray  from  a  collection  of  images. 

The  next  step  in  our  software  is  to  perform  a  simple  registration  procedure  to  get  the  coordinate  frame  transformation 
from  the  three-dimensional  AURORA  coordinates  to  the  two-dimensional  image  space  on  the  computer  screen.  From 
this  point  through  the  rest  of  the  procedure,  it  is  necessary  to  keep  the  patient  from  moving  on  the  table.  With  the  mouse 
pointer,  the  user  is  asked  to  click  on  the  fiducial  points  on  the  screen  that  he  intends  to  use.  Basically  any  points  on  the 
body  that  can  be  readily  identified  on  both  the  chest  x-ray  and  the  outside  of  the  patient’s  body  can  be  used  as  fiducial 
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points.  We  recommend  using  the  sternal  notch,  xiphoid  process,  and  the  acromioclavicular  junction  on  each  side 
(shoulders)  as  four  easily  found  fiducial  points  (Figure  2).  At  least  three  points  are  necessary  for  registration,  but  the 
physician  may  use  as  many  as  he  chooses. 


The  physician  then  touches  the  tip  of  the  catheter  to  the 
corresponding  fiducial  points  on  the  patient’s  body. 
When  this  is  complete,  the  result  is  the  frame 
transformation  from  patient  space  to  image  coordinates. 
At  this  point,  the  user  is  ready  to  enable  continuous 
tracking  of  the  catheter.  The  physician  can  then  move 
the  catheter  aroimd  outside  fte  patient’s  body  to 
determine  if  the  registration  is  accurate  enou^  to 
continue. 

Although  the  registration  from  any  single  patient  to  a 
generic  chest  x-ray  will  obviously  not  be  perfect  in 
cases  where  a  patient-specific  chest  x-ray  is  not 
available,  the  physician  still  can  qualitatively  make  the 
decision  whether  the  PICC  line  is  placed  correctly  or 
incorrectly  in  most  cases.  If  the  registration  from  the 
patient  to  the  generic  image  is  very  poor,  then  the 
physician  has  the  option  of  either  getting  a  chest  x-ray 
of  the  patient,  or  simply  inserting  the  PICC  line  the 
traditional  way. 


The  physician  is  now  able  to  begin  inserting  the  catheter.  The  time  it  takes  to  do  the  pivot  calibration  and  registration  is 
actually  very  short.  The  entire  process  can  be  done  in  less  than  five  minutes.  With  continuous  tracking  enabled,  the 
catheter  is  displayed  on  the  screen  as  a  pointer  which  rotates  to  show  the  orientation  of  the  tip.  As  the  catheter  is  moved 
in  the  patient,  the  software  continuously  updates  the  position  and  orientation,  showing  the  entire  trail  of  the  catheter  path. 
If  the  screen  becomes  too  cluttered,  the  user  may  click  on  a  button  to  clear  the  screen  and  start  displaying  the  pointers  on 
a  fresh  image.  The  position  of  the  catheter  tip  is  shown  in  two  dimensions  in  the  plane  of  the  x-ray.  Since  most  of  the 
motion  of  the  catheter  will  be  in  that  plane,  and  since  the  physician  is  already  familiar  with  that  view  from  the 
fluoroscope,  we  believe  that  this  will  be  a  good  enough  visualization  for  the  procedure.  The  orientation  angle  is  shown  in 
three  dimensions,  with  a  visualization  of  the  catheter  going  into  or  out  of  the  screen  when  appropriate. 

3.  RESULTS 

The  software  was  evaluated  qualitatively  with  a  plastic  model  of  a  human  torso  to  determine  if  the  display  would  give  a 
good  enough  visualization  to  be  useful.  The  phantom  consisted  of  a  life-size  plastic  model  of  a  human  arm  and  chest 
with  tubing  to  simulate  the  vasculature.  The  plastic  ‘veins’  contained  a  junction  of  three  different  paths  —  down  towards 
the  heart,  across  the  chest,  and  up  towards  the  neck  to  simulate  correct  and  ^vrong  paths  for  the  catheter.  Six  BB  fiducials 
for  registration  were  placed  on  the  phantom  and  a  chest  x-ray  was  obtained  of  the  model.  Turns  into  wrong  veins  were 
easily  seen  in  the  display  without  any  problem.  In  addition,  we  partially  blocked  a  vein  to  simulate  the  catheter  getting 
stuck,  which  was  also  detected  immediately. 

The  graphical  user  interface  for  the  program  is  shown  in  Figure  3.  The  steps  in  the  planning  and  treatment  process  are 
indicated  by  the  pushbuttons  on  the  right  hand  side  of  the  screen.  The  options  for  loading  images  and  saving  data  are 
located  in  the  pulldown  menus  at  the  top  of  the  screen.  The  center-left  shows  the  loaded  x-ray  image  of  our  plastic  torso 
phantom  and  is  the  main  display  window.  The  catheter  pointer  displays  within  the  limits  of  this  image  window.  The  tip 
of  the  catheter  is  displayed  as  a  magenta  circle,  with  the  orientation  represented  by  light  blue  trailing  lines.  To  create  the 
catheter  path  display  shown  below,  we  threaded  our  catheter  through  the  plastic  tubing  representing  veins  from  the 
model’s  arm  through  to  the  chest. 


Figure  2:  Fiducial  points  on  a  sample  chest  x-ray.  The  points 
correspond  to  the  sternal  notch,  xiphoid  process,  and  shoulder 
bones. 
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Figure  3:  Graphical  user  interface  with  sample  display  of  a  catheter  path. 


The  software  was  evaluated  quantitatively  for  accuracy  by  the  following  method.  We  created  an  evenly  spaced  grid  of 
nine  points,  and  followed  through  the  entire  procedure  for  using  our  program.  We  first  performed  five  pivot  calibrations 
to  determine  the  variability  between  trials.  The  variability  was  found  to  be  negligible  (<  0.2  mm).  We  then  went  through 
registration  with  all  nine  points  and  started  continuous  tracking.  We  defined  the  positional  error  of  the  system  by  the 
physical  distance  corresponding  to  the  difference  between  where  the  pointer  is  displayed  on  the  screen  and  the  point  on 
the  image  where  it  should  actually  be  displayed.  In  other  words,  this  corresponds  to  the  difference  between  where  the 
physician  thinks  the  catheter  is  and  where  the  catheter  actually  is  in  space.  We  calculated  the  orientation  error  by  testing 
a  set  of  eight  angles  (0,  45,  90...).  We  oriented  the  catheter  at  these  angles  and  reported  error  as  the  difference  between 
these  and  the  measured  angles  of  orientation.  We  feel  that  this  gives  a  more  useful  assessment  of  the  accuracy  of  the 
system  than  a  simple  registration  error  calculation.  From  five  trials,  we  got  an  average  positional  error  of  1.5±1.1  mm 
and  an  average  orientation  error  of  2.611.3  degrees  in  the  plane  of  the  x-ray  image.  We  believe  that  this  accuracy  is 
sufficient  for  the  application. 


4.  DISCUSSION 

As  the  reliability  and  quality  of  electromagnetic  tracking  systems  increases,  electromagnetic  navigation  has  more 
potential  for  providing  guidance  in  minimally  invasive  procedures.  While  some  research  groups  have  chosen  to  develop 
its  application  in  more  complex  procedures  such  as  cardiac  arrhythmia  surgery,  we  have  chosen  to  research  its  feasibility 
in  a  more  simple  application.  With  this,  we  hope  to  show  that  electromagnetic  tracking  can  easily  be  integrated  into 
image-guided  interventions.  The  Lucent  Medical  Systems  Corporation  developed  a  method  for  electromagnetically 
tracking  PICC  and  CVC  catheters  in  1998.  However,  it  differs  fi’om  our  research  in  several  key  points. 
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Their  system  consists  of  a  small  1mm  magnet  implanted  in  the  tip  of  a  catheter, 
and  a  handheld  display  containing  the  magnet  detectors.  The  handheld  display  is 
moved  above  the  patient  as  the  catheter  is  inserted,  and  the  location  of  the 
catheter  is  displayed  on  a  screen  on  the  device  (Figure  4).  This  system,  however 
only  shows  the  current  location  of  the  catheter  tip,  not  the  whole  path  history. 

Ours  tracks  the  catheter  more  directly  in  3D  coordinates  relative  to  the  patient, 
while  their  system  only  gives  the  2D  location  relative  to  the  handheld  sensor. 

We  have  not  taken  the  research  to  the  next  step  of  animal  trials  for  the  following 
reasons.  A  guidewire  and  catheter  combination  incorporating  an  electromagnetic 
fiducial  was  not  available  at  the  time  of  this  work.  In  the  real  procedure  a 
guidewire  must  first  be  placed  independent  of  the  catheter  to  establish  the  correct 
length  of  the  catheter  for  each  individual  patient.  In  addition,  even  if  we  were  to 
try  this  using  a  catheter  alone  to  demonstrate  the  tracking  concept,  the  current  catheter  is  not  steerable  enough  for  such  a 
procedure.  The  alternative  electromagnetic  navigation  system  described  above  has  been  successfully  tested  in  pigs  [3]. 
In  addition,  our  research  group  has  done  similar  work  to  use  electromagnetic  tracking  in  needle  placement  in  swine  and 
cadaver  studies  [5]. 

While  we  limited  our  research  specifically  to  PICC  line  insertion,  the  software  we  have  developed  is  general  enough  that 
it  can  be  easily  applied  to  other  similar  applications  such  as  CVC  placement,  bronchoscopy,  colonoscopy,  and 
potentially  any  application  where  it  would  be  useful  to  track  the  location  of  a  catheter  in  the  human  body. 

5.  CONCLUSIONS 

This  paper  presented  a  system  to  use  an  electromagnetic  guidance  system  to  track  the  catheter  in  PICC  line  placement. 
We  believe  that  this  type  of  system  has  broad  applicability  to  other  procedures  and  may  become  more  widely  used  in  the 
future.  Optical  tracking  has  already  gained  wide  acceptance  as  a  good  method  for  tracking  instruments  outside  the 
human  body.  Electromagnetic  tracking  provides  a  method  that  may  be  able  to  replace  x-ray  guidance  in  a  wide  variety  of 
procedures.  Our  system  could  potentially  be  used  at  the  bedside  without  real-time  fluoroscopic  guidance  (only  a  final 
confirming  x-ray  would  be  needed)  and  free  up  fluoroscopy  equipment  for  other  procedures. 
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for  catheter  tracking  (from 
www.lucentmedical.com/zortran.htm) 
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Breakdown  of  Tracking  Accuracy  for  Electromagnetically  Guided  Abdominal 
Interventions 

Jonathan  Tang*,  Kevin  Cleary  * 
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Georgetown  University  Medical  Center,  2115  Wisconsin  Ave.,  #603,  Washington,  DC 
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This  study  investigates  the  sources  of  error  for  an  image-guided  surgery  system 
based  on  electromagnetic  tracking.  We  characterized  fiducial  localization  error  in  both 
magnetic  and  image  space,  modeled  error  propagation  using  a  Monte  Carlo  simulation, 
and  estimated  system  errors  for  registration  schemes  with  minimal  human  error. 
Simulation  results  indicated  that  we  would  obtain  much  closer  fiducial  registration  if  we 
were  to  have  a  system  with  automatic  fiducial  localization. 

1.  INTRODUCTION 

1.1  Purpose  of  Study 

The  purpose  of  this  study  was  to  investigate  the  error  sources  in  an  image-guided 
surgery  system  based  on  electromagnetic  tracking  that  we  been  developing  for  minimally 
invasive  abdominal  interventions.  This  breakdown  will  enable  us  to  better  understand  the 
effects  of  various  components  on  the  system  accuracy. 

Over  the  past  two  years,  oiu  research  group  has  developed  an  image-guided 
surgery  system  [1]  incorporating  a  beta  version  of  the  AURORA  magnetic  tracker 
(Northern  Digital,  Waterloo,  Canada).  In  this  system,  registration  is  done  based  on  pre¬ 
operative  CT  scans  and  a  set  of  fiducials  that  can  be  localized  in  both  image  and  magnetic 
space.  The  AURORA  uses  small  sensor  coils  that  can  be  embedded  into  the  tip  of 
needles  or  catheters  to  directly  track  their  position  and  orientation.  The  system  allows  the 
physician  to  accurately  guide  a  magnetically  tracked  needle  into  the  abdominal  area  for  a 
variety  of  interventional  procedures  that  are  usually  done  under  x-ray  fluoroscopy.  The 
system  has  been  tested  on  phantom  torsos,  respiratory  motion  simulators,  and  cadavers, 
all  of  which  have  been  rigid-body  approximations  of  true  physiological  conditions.  In  a 
liver  phantom,  the  physician  using  the  system  has  been  able  to  target  simulated  liver 
tumors  with  successful  placement  in  14  of  16  attempts.  However,  the  system  does  not 
perform  as  well  with  targets  smaller  than  1cm,  deeper  targets,  and  various  other 
experimental  conditions.  This  prompted  us  to  investigate  the  error  soiuces  of  the  system. 

The  goals  of  this  paper  are: 

•  To  characterize  the  overall  fiducial  localization  error  in  both  magnetic  space  and 
image  space  for  oiu  image-guided  intervention  system  (IGS). 

•  To  model  error  propagation  in  our  IGS  using  a  Monte  Carlo  simulation. 

•  To  approximate  the  contribution  of  human  error  towards  fiducial  registration 
error  and  to  simulate  registration  schemes  without  the  influence  of  human  error. 

1.2  Registration  Error  Background 

A  common  measure  of  overall  fiducial  misalignment  is  the  root-mean-square  error  called 
fiducial  registration  error  (FRE)  [2].  The  fiducial  registration  error  is  the  number  that 
most  image-guided  systems  return  after  registration  of  coordinate  systems.  Fiducial 
misalignment  stems  from  the  inability  to  correctly  determine  fiducial  locations  in 
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magnetic  space  and  image  space.  The  determination  of  a  precise  point  within  a  fiducial  is 
called  fiducial  localization.  The  inevitable  displacement  of  the  localized  point  from  the 
correct  position  is  called  the  fiducial  localization  error  (FLE).  Such  error  occurs  in  both 
image  and  magnetic  space.  Misalignment  of  a  target  point  due  to  registration  is  called 
Target  Registration  Error  (TRE).  For  an  in-depth  mathematical  review  of  FLE,  FRE,  and 
TRE,  please  consult  [2]. 

2.  METHODS 

2.1  System  Implementation 

Figure  1  below  shows  the  implementation  of  the  system  as  recently  used  in  a  swine  study 
in  the  interventional  suite.  The  goal  of  this  study  was  to  hit  simulated  lesions  in  a  swine 
liver  using  pre-procedure  CT  scans  and  image  guidance  based  on  electromagnetic 

tracking.  The  physician  watches  the 
image  overlay  shown  in  the  lower  left 
of  the  figure  to  manipulate  the  needle 
along  a  pre-planned  path.  Respiration 
is  accounted  for  using  another 
electromagnetically  tracked  needle 
(MagTrax,  Traxtal  Technologies, 
Bellaire,  TX)  that  was  placed  in  the 
liver  prior  to  the  CT  scan.  The  current 
system  had  mixed  success  in 
accurately  hitting  the  targets,  which 
prompted  us  to  investigate  the  sources 
of  error  described  in  this  paper. 

Figure  1  -  Swine  Study  in  the  Interventional  Suite 

2.2  Error  Breakdown 

The  main  sources  of  FLE  in  our  rigid  model  include  errors  in  magnetic  space, 
image  space,  and  human  error  in  physical  space.  The  diagram  below  depicts  the 
propagation  of  error  from  magnetic  and  image  space  to  fiducial  registration  and  target 
registration. 


Figure  2  -  Accuracy  breakdown  of  our  registration  process 
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We  categorized  the  error  sources  into  three  main  groups:  1)  magnetic  space,  2)  CT  space, 
and  3)  physical  space. 

1)  Magnetic  space.  The  most  obvious  error  source  is  the  inherent  accuracy  of  the 
magnetic  tracking  system.  We  use  circular  multi-modality  fiducials  (IZI  Corporation, 
Baltimore,  MD)  as  skin  fiducials  in  our  registration  process.  For  the  experiment 
described  in  this  paper,  we  built  a  custom  plexiglass  registration  phantom  using  0.8mm 
tantalum  beads  (Tilly  Medical,  Sweden)  as  fiducials.  To  localize  a  fiducial  in  magnetic 
space,  the  center  of  the  fiducial  is  touched  with  a  magnetically  tracked  probe.  Error 
sources  here  include  the  placement  error  in  manually  localizing  the  center  of  the  fiducial 
and  error  in  calculating  the  location  of  the  probe  tip  from  die  embedded  sensor  coil, 
which  involves  the  combined  translational  and  orientation  error  of  the  tracking  system. 

2)  CT  space.  There  will  be  some  error  inherent  to  the  CT  scanner  and  image 
reconstruction  process  but  this  is  considered  to  be  fairly  small  as  we  are  using  1  mm 
slices  in  our  experimental  studies.  Most  of  the  error  in  localizing  image  fiducials  will 
result  from  our  manual  process  of  choosing  the  center  of  the  fiducials  on  the  CT  scan. 

3)  Physical  space.  In  any  registration  procedure  where  manual  intervention  is  required  to 
select  or  localize  the  fiducials,  there  will  always  be  human  error.  In  our  system,  human 
error  exists  at  the  step  where  we  localize  the  skin  fiducials  in  magnetic  space  and  image 
space;  when  the  user  taps  out  the  centers  of  the  skin  fiducials  on  the  patient  and  clicks  on 
the  center  of  the  image  fiducials  on  the  CT  scan.  This  error  can  not  be  directly  measured 
independently  fi’om  errors  in  the  rest  of  om  system.  Errors  in  magnetic,  image  and 
physical  space  influenced  the  system’s  ability  to  accurately  localize  fiducials.  These  were 
considered  fiducial  localization  errors  (FEE)  which  contribute  to  fiducial  registration 
accuracy  (measured  as  fiducial  registration  error).  From  earlier  work  of  Fitzpatrick  et  al. 
[2],  we  know  that  FEE  is  a  contributor  to  targeting  accuracy,  measured  as  target 
registration  error,  along  with  the  configuration  and  number  of  fiducials,  and  the  distance 
of  the  target  from  the  centroid  of  the  fiducial  configuration. 

2.3  Measurement  of  Error 

To  better  understand  and  quantify  the  contributions  of  some  of  these  error  sources,  we 
conducted  the  following  experiments: 

•  Experiment  1  examined  the  accuracy  of  the  AURORA 

•  Experiment  2  investigated  the  offset  from  the  coil  position  to  the  needle  tip 

•  Experiment  3  measured  fiducial  localization  error  using  a  plexiglass  phantom 

•  Experiment  4  modeled  error  propagation  using  a  Monte  Carlo  simulation 

Experiment  1;  In  previous  work,  the  accuracy  of  the  AURORA  magnetic  tracker  was 
characterized  in  both  translational  and  orientation  aspects.  In  those  experiments  we  used 
a  novel  six  degree  of  freedom  robot  to  serve  as  a  precision  positioning  device.  To 
measure  positional  accuracy,  the  robot  was  programmed  to  accurately  place  the 
magnetically  tracked  probe  at  7  different  positions  over  a  100x40x40mm  volume  [3].  For 
orientation  accuracy,  the  robot  was  programmed  to  position  the  probe  at  the  8  comers  of 
a  lOxlOxlOmm  volume,  where  the  probe  was  oriented  from  -30  to  30  degrees  in  10 
degree  increments  [4].  The  robot  was  considered  accurate  and  ground  trath  from 
previous  studies. 
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Results  1:  Within  the  100x40x40  mm  volume,  the  AURORA  was  accurate  to  better  than 
1  mm  [3]  in  translational  position.  The  orientational  study  was  done  at  the  center  of  the 
AURORA  workspace  and  foimd  an  orientation  accuracy  of  better  than  1  degree  [4].  It 
should  be  noted  that  these  were  preliminary  studies  and  a  more  definitive  study  with 
more  trials  and  a  larger  workspace  remains  to  be  done. 

Experiment  2:  To  measure  the  accuracy  in  calculating  the  position  of  the  probe  tip,  we 
used  the  same  robot  and  programmed  it  to  pivot  our  magnetically  tracked  probe  at  its  tip. 
Figure  3  below  shows  the  robot  holding  the  needle/probe  for  a  pivot  routine.  This 
technique  is  the  well  known  pivot  routine  that  is  used  for  calculating  offsets  between 
tracked  sensors  and  the  tip  of  the  localization  probe.  From  the  position  and  orientation 
data  collected,  we  were  able  to  calculate  the  offset  between  the  magnetic  tracking  coil 
and  the  tip  of  the  probe.  Using  the  offset,  calculated  in  the  least  squares  sense,  we  were 
able  to  determine  the  position  of  the  tip  given  the  position  and  orientation  of  the  magnetic 
coil  in  the  probe  as  returned  by  the  AURORA  system.  The  error  in  determining  tip 
position  was  measured  as  the  variance  in  the  calculated  tip  position  during  several  robot 

Results  2:  The  error  in  determining  tip  position  was 
measured  as  standard  deviations  ranging  fi-om  0.2  mm  to  0.5 
mm  depending  on  the  speed  of  the  pivot,  the  axis  of  rotation, 
and  the  needle  used.  We  found  that  larger  angle  pivots  done 
at  slower  speeds  result  in  lower  error. 


Figure  3  -  Robot  doing  pivot  with  sample  needle 

Experiment  3:  To  measure  fiducial  localization  error  in  both  magnetic  and  image  spaces, 
we  ran  the  registration  procedure  twenty-five  times  on  our  custom  built  plexiglass 
phantom,  recording  positions  of  both  magnetic  and  image  fiducials  and  the  fiducial 
registration  error.  The  phantom  was  constructed  out  of  a  10x10x15cm  solid  block  of 
plexiglass,  where  each  side  had  five  divot  points  with  a  0.8mm  tantaliun  bead  embedded 
at  each  divot  point.  Four  holes  were  drilled  in  the  block  to  embed  tracked  needles  used  as 
internal  fiducials. 

tfiSrriiyf'r.r* 
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In  addition,  we  did  fifteen  trials  of  registration  on  our  torso  model  using  multi-modality 
skin  fiducials,  but  found  no  statistically  significant  difference  in  registration  results 
compared  to  using  tantalum  beads. 

Results  3:  We  conducted  twenty-five  registration  trials  using  this  plexiglass  phantom. 
The  distributions  of  measured  fiducials  at  each  divot  are  best  described  as  point  clouds 
centered  about  fiducial  position  ground  truth.  The  fiducial  localization  error  was 
measured  as  the  variance  of  each  point  cloud.  The  distribution  of  fiducials  in  magnetic 
space  had  a  range  of  standard  deviations  between  0.23  and  2.24  mm  with  an  average  of 
0.8mm.  The  distribution  of  fiducials  in  image  space  had  a  standard  deviation  of 
approximately  0.3  mm  in  the  axial  plane  and  0.5mm  between  planes.  We  found  that  the 
average  fiducial  registration  error  for  25  manual  registration  trials  was  1.1mm  ranging 
between  0.7mm  and  1.3mm  with  a  standard  deviation  of  0.21mm.  The  internal  fiducial, 
which  was  magnetically  tracked  and  did  not  require  any  manual  localization,  had  a  very 
small  standard  deviation  of  0.05mm. 

Experiment  4;  Using  measurements  of  FLE  fi*om  experiment  3,  we  modeled  error 
propagation  during  the  registration  procedure  with  a  Monte  Carlo  simulation  done  in 
Matlab.  We  assumed  that  the  FLE  of  each  fiducial  is  an  independent  identically 
distributed  variable  that  can  be  modeled  by  a  normal  distribution  with  standard  deviation 
0.4mm  for  image  fiducials  and  0.8mm  for  magnetic  fiducials.  Using  randomly  perturbed 
fiducial  locations,  we  calculated  the  best  point-based  rigid  registration  in  the  least  squares 
sense.  This  calculation  was  done  with  10  000  different  perturbations  as  a  Monte  Carlo 
simulation.  An  example  of  the  output  is  shown  in  Figure  7b.  Using  this  simulation,  we 
were  able  to  estimate  the  effect  of  different  FLE  on  the  registration  error. 


FRE  (huiO  FRE  (mm) 


Figure  7a  -  experimental  data  Figure  7b  -  simulation  results 

Results  4:  During  simulation  trials,  using  0.4mm  as  the  standard  deviation  for  image 
fiducial  FLE,  and  0.8mm  as  the  standard  deviation  for  magnetic  fiducial  FLE,  we  find  the 
average  of  FRE  to  be  1.17  +/-  0.28mm.  (Figure  7b)  This  is  consistent  with  experimental 
data  (1.1  +/-  0.21mm)  from  twenty-five  trials  done  in  experiment  3.  (Figure  7a)  To 
simulate  computerized  selection  of  image  fiducials,  we  used  O.Omm  as  the  standard 
deviation  for  image  fiducial  FLE,  and  0.8nim  for  magnetic  fiducials.  We  chose  to  use  a 
standard  deviation  of  O.Omm  for  image  fiducials  because  computerized  selection  of 
fiducials  results  in  the  same  coordinates  every  time.  For  that  simulation,  we  get  a 
registration  error  of  1 .0  +/-  0.25nim.  To  estimate  the  registration  error  when  the 
procedure  has  an  automatic  localization  process,  we  changed  the  parameters  of  the 
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simulation  such  that  it  would  calculate  registration  with  FLE  equal  to  the  internal  tracked 
fiducial  (~  O.OSmm).  This  effectively  simulates  automatic  localization  of  skin  fiducials  as 
if  they  were  magnetically  tracked  fiducials  on  the  skin  surface.  Registration  error  was 
predicted  to  be  0.54  +/-  0.13mm.  This  shows  that  we  can  reduce  most  of  the  registration 
error  if  we  automatically  localize  our  skin  fiducials. 

3.  CONCLUSION 

There  are  many  factors  that  can  contribute  to  the  overall  accuracy  of  an  image-guided 
surgery  system  based  on  magnetic  tracking.  A  systematic  analysis  is  necessary  to  isolate 
critical  factors  and  determine  how  optimal  performance  can  be  achieved.  We  believe  that 
the  greatest  contributor  to  registration  error  is  fi*om  the  manual  fiducial  localization 
process.  Our  simulation  indicated  that  if  we  were  to  have  a  system  with  automatic 
fiducial  localization,  we  would  obtain  much  closer  fiducial  registration. 
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1.  ABSTRACT 

In  this  Study,  we  collected  respiratory  motion  data  of  external  skin  markers  and  internal  liver  fiducials  from  several 
swine.  The  POLARIS  infrared  tracking  system  was  used  for  recording  reflective  markers  placed  on  the  swine’s 
abdomen.  The  AURORA  electromagnetic  tracking  system  was  used  for  recording  2  tracked  needles  Implanted  into 
the  liver.  This  data  will  be  used  to  develop  correlation  models  between  external  skin  movement  and  internal  organ 
movement,  which  is  the  first  step  towards  the  ability  to  compensate  for  respiratory  movement  of  the  lesion.  We  are 
also  developing  a  motion  simulator  for  validation  of  our  model  and  dose  verification  of  mobile  lesions  in  the 
CYBERKNIFE  Suite.  We  believe  that  this  research  could  provide  significant  information  towards  the  development  of 
precise  radiation  treatment  of  mobile  target  volumes. 

Keywords:  Respiratory  Motion  Tracking,  Cyberknife 

2.  PURPOSE 

Radiation  treatment  of  mobile  lesions  has  been  a  long  standing  challenge  in  radiation  therapy.  The  ability  to 
compensate  for  respiratory  motion  of  internal  organs  is  highly  desirable  in  stereotactic  radiosurgery.  Previous 
researchers  have  shown  that  it  may  be  possible  to  predict  internal  organ  motion  using  external  skin  marker 
information.^  The  purpose  of  this  study  was  to  collect  respiratory  motion  data  of  skin  markers  and  internal  fiducials 
from  several  swine  studies.  We  hope  to  use  this  data  to  implement  and  evaluate  a  correlation  algorithm  between 
external  skin  motion  and  internal  organ  motion.  We  are  also  developing  a  motion  simulator  device  to  reproduce  these 
respiratory  motions  for  evaluation  of  the  correlation  model.  This  motion  simulator  can  also  be  used  for  dose 
verification  of  mobile  lesions.  Our  research  goal  is  to  develop  methods  for  precise  radiation  treatment  of  mobile  target 
volumes. 


3.  METHODS 

The  CYBERKNIFE  (Accuray,  Sunnyvale  CA)  is  a  radiation  treatment  system  that  incorporates  a  linear  accelerator 
mounted  on  a  robotic  arm  (Figure  1).  It  is  able  to  track  tumor  location  using  two  orthogonal  x-ray  cameras  that  track 
fiducials  inserted  before  radiation  treatment.  The  system,  as  currently  installed  at  Georgetown  University  Hospital, 
can  automatically  compensate  for  up  to  10  mm  of  patient  movement  but  only  once  every  30  to  60  seconds.  Since 
robotic  control  of  the  linear  accelerator  provides  the  unique  possibility  to  do  real-time  tumor  motion  tracking  during 
stereotactic  radiosurgery,  Accuray  has  developed  SYNCHRONY,  a  real-time  tracking  system  that  aims  to  precisely 
track  lesions  in  organs  with  significant  respiratory  motion.  With  SYNCHRONY,  a  correlation  model  between  skin 
motion  and  internal  organ  motion  is  established  by  tracking  skin  motion  with  infrared  markers  and  tracking  implanted 
fiducials  with  two  orthogonal  x-ray  cameras.  The  CYBERKNIFE  robotic  arm  follows  the  internal  organ  motion  based 
on  the  prediction  from  the  skin  motion.  To  validate  this  technology,  there  exists  a  need  for  external  and  internal 
respiratory  motion  data  obtained  at  fast  rates  and  a  motion  simulator  platform  to  replicate  the  respiratory  motion. 


*Contact  information:  Jonathan  Tang  itang@isis.imac.georgetown.edu.  phone  (202)-687-2902 
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Figure  3a:  (left)  Polaris  Optical  Tracking  System. 

Figure  3b:  (right)  Polaris  Passive  and  Active  Marker 

Previously  our  group  has  shown  the  feasibility  of  fast  update  tracking  of  external  fiducials  in  the  CYBERKNIFE  suite^ 
(Figure  2,  Figure  Sab).  Using  the  POLARIS  optical  tracker  (Northern  Digital  Inc,  Waterloo,  Canada),  we  tracked  four 
reflective  skin  makers  placed  on  patients’  chests  during  radiosurgery  treatment  in  the  CYBERKNIFE  suite  under  an 
IRB  approved  protocol.  That  data  has  proven  useful  in  studying  external  skin  motion  of  human  patients,  but  does  not 
provide  information  about  the  movement  of  internal  fiducials.  In  this  paper,  we  collected  positions  of  both  external 
and  internal  fiducials  in  swine  to  provide  data  for  respiratory  motion  compensation  research. 
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Figure  5  -  Tracking  of  Internal  fiducials  and  External  skin  markers  in  swine 


We  obtained  approval  from  the  Georgetown  Animal  Care  and  Use  Committee  to  track  external  and  internal  fiducials  in 
four  swine.  The  POLARIS  infrared  tracking  system  was  used  for  recording  4  passive  reflective  markers  placed  on  the 
swine’s  abdomen.  The  AURORA  electromagnetic  tracking  system  (Northern  Digital)  was  used  for  recording  2 
electromagnetically  tracked  needles  (Traxtal  Technologies,  Bellaire,  Texas)  implanted  into  the  liver.  The  AURORA 
field  generator,  the  2  needles,  and  the  4  passive  markers  are  shown  in  Figure  5. 

In  order  to  register  the  AURORA  to  the  POLARIS  coordinate  system,  we  localized  the  position  of  the  POLARIS 
optically  tracked  skin  markers  with  the  AURORA  tracked  needles  by  removing  the  reflective  sphere  ball  and  pointing 
to  the  plastic  stem  presumed  to  be  in  the  center  of  the  reflective  sphere.  We  can  solve  for  the  transformation  matrix 
that  registers  both  coordinate  systems,  given  the  positions  of  the  four  markers  in  both  POLARIS  and  AURORA  space. 

Fiducial  location  data  was  recorded  at  10  hertz  for  external  skin  marker  data,  and  30  hertz  for  internal  fiducial  data,  for 
a  duration  of  four  minutes  with  the  swine  ventilated  at  twelve  breaths  per  minute. 

4.  RESULTS 

The  first  two  columns  of  Table  1  show  the  position  of  the  skin  fiducials  localized  in  both  POLARIS  and  AURORA 
coordinate  systems.  This  data  was  taken  from  the  most  recent  swine  study,  and  has  been  rounded  to  the  first  decimal 
place.  Column  1  lists  the  skin  markers  as  measured  with  the  POLARIS  tracker,  column  2  lists  the  same  skin  markers 
as  measured  by  the  AURORA  tracker.  We  registered  the  two  coordinate  systems  together  and  applied  the  resulting 
frame  transformation  to  the  skin  markers  measured  in  AURORA  coordinate  space.  Column  3  is  the  result  of 
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transforming  the  skin  markers  from  AURORA  to  POLARIS  coordinate  space.  The  transformation  matrix  can  be  used 
to  transform  any  coordinate  measured  in  AURORA  space  into  corresponding  POLARIS  space. 


Table  1;  Position  of  Skin  Fiducials  From  Most  Recent  Study  (distances  in  mm) 


Skin  Markers 
(POLARIS  space) 

Skin  Markers 
(AURORA  space) 

Transformed  Skin  Markers 
(AURORA  to  POLARIS  space) 

El 

338.8,  56.6,  -1467.8 

-109.9, 112.8,-216.8 

338.4,  57.6,  -1464.4 

B. 

363.5,-1.7,  -1424.4 

-61.1,  120.3,  -156.2 

365.1,-1.5,-1420.7 

c. 

339.8  13.8-1511.6 

-100.3, 42.7,  -214.8 

338.9, 15.0,-1520.9 

D. 

352.3,  -36.9,  -1477.2 

-65.5,  53.1,  -152.2 

352.0,  -39.4,  -1475 

The  registration  error  for  this  transformation  was  5.7mm.  Registration  error  is  calculated  as  the  root  mean  square  error 
between  transformed  AURORA  fiducial  positions  and  POLARIS  fiducial  positions.  Figure  6  shows  the  first  ten 
seconds  of  data  from  one  external  and  one  internal  fiducial.  Frequency  analysis  indicates  that  internal  and  external 
fiducial  movement  is  in  phase  with  the  largest  frequency  component  between  0.18  and  0.21  hertz  (equivalent  to  1 1  to 
12.5  breaths  per  minute)  as  shown  in  the  FFT  analysis  of  the  POLARIS  data  in  Figure  7. 


Figure  6:  Ten  seconds  of  respiratory  motion  for  both  external  and  internal  fiducials. 


Page  57 


To  be  published  in  SPIE  Medical  Imaging  2004  proceedings 


Figure  7  -  Skin  motion  FFT  showing  main  frequency  component  at  11  breaths  /  minute. 


The  data  from  these  animal  studies  was  used  to  program  a  respiratory  motion  simulator  customized  for  the 
CYBERKNIFE  Suite  at  Georgetown  University  Hospital.  The  respiratory  motion  simulator  consists  of  two 
independent  and  programmable  three-dimensional  motion  platforms  built  completely  with  off-the-shelf  components. 
(Figure  Sab)  Wiring,  assembly  and  software  development  were  done  in-house  at  Georgetown  University.  The 
components  were  selected  based  on  the  technical  requirements  of  motion  amplitude  and  frequency,  which  were 
established  from  real  patient  data  taken  at  Georgetown  University  Hospital.  The  platforms  are  controlled  by  a  PC 
through  an  Ethernet-addressable  motion  control  board.  (Galil  Motion  Control,  Inc.)  See  Figure  8a  for  a  picture  of  the 
respiratory  motion  simulator  in  the  CYBERKNIFE  Suite. 

The  simulator  is  able  to  precisely  simulate  the  three-dimensional  respiratory  motion  of  skin  and  internal  organ  fiducial 
markers.  This  motion  simulator,  along  with  external  and  internal  fiducial  motion  gathered  from  animals  in  this  study, 
will  be  used  to  assist  in  implementing  and  evaluating  correlation  algorithms  between  external  skin  motion  and  internal 
organ  motion.  This  motion  simulator  can  also  be  used  for  dose  verification  of  mobile  lesions  in  the  CYBERKNIFE 
Suite. 
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5.  CONCLUSION 

In  this  study,  we  have  collected  respiratory  motion  data  from  several  swine  using  the  POLARIS  optical  tracker  and 
AURORA  magnetic  tracker.  This  will  be  used  to  develop  correlation  models  between  external  skin  movement  and 
internal  organ  movement,  which  is  the  first  step  towards  the  ability  to  compensate  for  respiratory  movement  of  the 
lesion.  We  are  also  developing  a  motion  simulator  for  validation  of  our  model  and  dose  verification  of  mobile  lesions 
in  the  CYBERKNIFE  Suite.  We  believe  that  this  research  could  provide  significant  information  towards  the 
development  of  precise  radiation  treatment  of  mobile  target  volumes,  which  is  an  extremely  important  clinical 
problem. 
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ABSTRACT 

We  are  developing  a  prototype  system  for  robotically  assisted  lung  biopsy.  For  directing  the  robot  in  biopsy  needle 
placement,  we  propose  a  non-invasive  algorithm  to  track  the  3D  position  of  the  target  lesion  using  2D  CT  fluoroscopy 
image  sequences.  A  small  region  of  the  CT  fluoroscopy  image  is  registered  to  a  corresponding  region  in  a  pre-operative 
CT  volume  to  infer  the  position  of  the  target  lesion  with  respect  to  the  imaging  plane.  The  registration  is  implemented  in 
a  coarse  to  fine  fashion.  The  local  deformation  between  the  two  regions  is  modeled  by  an  affine  transformation.  The 
sum-of-squared-differences  (SSD)  between  the  two  regions  is  minimized  using  the  Levenberg-Marquardt  method.  Multi¬ 
resolution  and  multi-start  strategies  are  used  to  avoid  local  minima.  As  a  result,  multiple  candidate  transformations 
between  the  two  regions  are  obtained,  from  which  the  true  transformation  is  selected  by  similarity  voting.  The  true 
transformation  of  each  frame  of  the  CT  fluoroscopy  image  is  then  incorporated  into  a  Kalman  filter  to  predict  the 
lesion’s  position  for  the  next  frame.  Tests  were  completed  to  evaluate  the  performance  of  the  algorithm  using  a 
respiratory  motion  simulator  and  a  swine  animal  study. 

Keywords:  motion  estimation,  real-time  tracking,  robust  tracking,  data  association,  textured  regions,  lung  biopsy, 
robotics 


1.  INTRODUCTION 

In  lung  biopsy,  the  position  of  the  target  lesion  may  vary  due  to  intrinsic  causes  such  as  respiratory  motion  or  extrinsic 
reasons  such  as  interactions  between  the  tissue  and  a  surgical  tool.  In  developing  a  robotic  system  to  assist  in  biopsy,  we 
need  to  provide  some  way  to  track  the  target  lesion.  CT  fluoroscopy  combines  the  advantages  of  both  CT  and 
fluoroscopy,  which  offers  an  opportunity  to  track  the  motion  of  the  target  lesion  in  real-time  during  the  intervention.  Our 
prototype  system  is  shown  in  Fig.  1 .  The  real-time  CT  fluoroscopy  image  is  captured  using  a  frame  grabber  (Accustream 
170,  Foresight  Imaging,  Lowell,  Massachusetts,  USA).  The  position  of  the  target  lesion  is  detected  from  the  image, 


Fig.  1  CT  fluoroscopy  guided  robotically  assisted  lung  biopsy 
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which  serves  as  input  for  the  robot  controller  to  compensate  for  the  motion.  This  paper  presents  a  motion  analysis 
method  to  estimate  the  3D  motion  of  a  target  lesion  using  the  CT  fluoroscopy  image  sequences.  The  method  aims  at 
tracking  the  lesion’s  motion  no  matter  whether  the  lesion  is  inside  or  outside  the  imaging  plane. 

Motion  tracking  is  a  well-established  area  of  research  in  computer  vision.  The  tracking  methods  can  be  broadly  classified 
into  feature-based  and  region-based  approaches.  The  feature-based  approaches  rely  on  the  extraction  of  a  sparse  set  of 
image  primitives  corresponding  to  distinctive  scene  features  and  the  matching  of  such  features  over  time  using  some 
form  of  search  procedure.  The  type  of  features  can  be  points,  lines,  edges,  comers  or  contours.  It  is  required  for  these 
approaches  that  the  features  being  tracked  stay  in  the  scene.  For  the  purpose  of  tracking  pulmonary  lesions  using  CT 
fluoroscopy,  the  available  features  in  the  CT  fluoroscopy  image  are  blood  vessels  and  bronchi.  Due  to  respiratory 
motion,  the  features  specified  in  one  frame  may  not  be  in  another  frame.  Therefore,  the  feature-based  approaches  cannot 
be  applied. 

The  region-based  approaches  make  direct  and  complete  use  of  all  available  image  intensity  information,  therefore 
eliminating  the  need  to  identify  and  model  a  special  set  of  features  to  track.  The  tracking  algorithm  of  this  paper  falls  into 
this  category.  The  algorithm  minimizes  the  sum-of-squared  differences  (SSD)  between  two  regions.  Variants  of  this 
method  have  been  presented  in  previous  work  by  other  researchers.  For  example,  Rehg  and  Witkin  [1]  tracked  affine 
deformations;  Hager  and  Belhumeur  [2]  reformulated  the  tracker  for  real-time  performance;  Shi  and  Tomasi  [3] 
connected  images  texture,  numerical  issues,  and  tracker  performance;  Szeliski  and  Coughlan  [4]  used  the  Levenberg- 
Marquardt  method  as  a  problem  solver  to  track  multi-bilinear  patches;  and  Gleicher  [5]  introduced  difference 
decomposition  to  solve  the  registration  problem  in  tracking,  where  the  difference  was  a  linear  combination  of  a  set  of 
basis  vectors. 

The  goal  of  this  tracking  algorithm  is  to  guide  a  surgical-assist  system  (a  needle  placement  robot).  Therefore,  the 
algorithm  must  be  robust  to  be  clinically  acceptable.  In  the  existing  literatures  on  robust  tracking,  Toyama  and  Hager  [6] 
presented  the  “Incremental  Focus  of  Attention”  architecture,  which  switches  between  algorithms  according  to  the  visual 
environment  to  achieve  robust  performance.  McCane  et.  al  [7]  presented  a  framework  for  merging  the  results  of 
independent  motion  trackers  using  a  classification  approach.  Chen  et.  al  [8]  used  neighborhood  relaxation  with  multi¬ 
candidate  pre-screening  to  robustly  track  image  regions. 


2.  METHODS 


Using  CT  fluoroscopy,  the  anatomy  in  the  imaging  plane  can  be  viewed  in  real-time.  Although  the  target  lesion  may  not 
always  stay  in  the  imaging  plane  due  to  respiratoiy  motion,  the  location  of  the  target  lesion  with  respect  to  the  imaging 
plane  can  be  estimated  from  a  pre-operative  CT  volume.  As  shovm  in  Fig.  2,  a  small  region  A  is  initially  selected  by  the 
physician  in  the  pre-operative  image  such  that  the  region  is  close  to  the  target  lesion  and  has  rich  texture.  If  this  region 
and  the  target  lesion  are  close  enough,  they  will  have  approximately  the  same  deformation.  By  registering  the  region  A 
to  its  corresponding  region  B  of  the  pre-operative  CT  volume,  the  location  of  the  target  lesion  with  respect  to  the 
imaging  plane  can  be  estimated  using  the  local  rigidity  around  the  lesion.  The  framework  of  the  algorithm  is  shown  in 
Fig.  3.  The  position  of  the  current  CT  fluoroscopy  region  in  the  CT  volume  is  predicted  from  previous  frames.  The  CT 
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fluoroscopy  image  region  is  then  registered  to  the  CT  volume  in  a  coarse-to-fme  fashion.  A  multi-start  strategy  is  used 
during  the  process,  so  multiple  results  are  generated.  The  true  motion  of  the  CT  fluoroscopy  region  is  selected  from  these 
candidate  results  by  similarity  voting.  The  selected  motion  vector  is  then  incorporated  into  a  Kalman  filter  to  predict  the 
region’s  position  for  the  next  frame. 

2.1  Coarse  registration 

The  purpose  of  the  coarse  registration  is  to  find  good  starting  points  for  the  fine  registration.  It  is  necessary  that  at  least 
one  of  these  starting  points  will  lead  to  the  solution  of  the  fine  registration.  Since  the  CT  fluoroscopy  image  has  a  large 
slice  thickness  (10  mm  in  our  case),  the  CT  volume  is  resampled  to  the  same  slice  thickness.  The  original  slice  interval  is 
preserved.  Although  the  adjacent  two  slices  have  overlap  in  the  resampled  CT  volume,  it  doesn’t  change  the  spatial 
position  of  each  slice.  Each  slice  of  the  resampled  CT  volume  is  then  downsampled  to  a  lower  resolution.  The  sample 
rate  is  determined  by  the  size  of  the  convergence  region  of  the  fine  registration  so  that  at  least  one  sampled  pixel  will  fall 
inside  the  basin  surrounding  the  minimum  of  the  SSD  residual.  Cross  correlation  is  performed  at  the  pixel  level  to 
estimate  the  relationship  between  the  CT  fluoroscopy  image  region  and  the  CT  volume.  The  pixels  best  correlated  with 
the  center  of  the  CT  fluoroscopy  image  region  are  selected  to  be  the  starting  points  of  the  fine  registration. 

2.2  Fine  registration 

The  lung  has  rich  textures  formed  by  blood  vessels  and  bronchi,  which  allows  the  use  of  local  information  to  register  the 
CT  fluoroscopy  image  region  to  the  CT  volume.  As  shown  in  Fig.  2,  by  registering  the  tissue  inside  the  region  to  the  pre¬ 
operative  CT  volume,  the  location  of  the  target  lesion  with  respect  to  the  imaging  plane  can  be  determined  using  the 
local  rigidity  around  the  lesion  in  the  CT  volume.  The  shape  of  the  region  can  be  either  a  ring  or  a  disk  in  this  research, 
depending  on  which  shape  encloses  more  texture  in  the  CT  fluoroscopy  image.  The  advantage  of  using  a  circular  region 
is  that  it  better  approximates  the  motion  of  the  region  with  respect  to  its  surrounding  area.  Before  the  registration,  the  CT 
volume  is  smoothed  by  Gaussian  filters.  Gradient  images  are  then  calculated.  The  tracking  algorithm  is  based  on 
minimizing  the  SSD  between  the  selected  region  in  the  CT  fluoroscopy  image  and  a  corresponding  one  in  the  CT 
volume.  The  Levenberg-Marquardt  method  [9]  is  used  to  solve  the  SSD  objective  function  with  a  good  starting  point. 
Compared  to  other  methods  for  solving  least  square  problems,  the  Levenberg-Marquardt  method  is  more  robust,  which 
usually  results  in  a  larger  convergence  region. 

The  CT  fluoroscopy  image  usually  has  a  large  slice  thickness.  As  shown  in  Fig.  4,  the  2D  CT  fluoroscopy  image  is 
actually  the  projection  of  all  the  tissue  inside  the  dashed  box  on  the  imaging  plane.  As  a  result,  the  CT  fluoroscopy  image 
cannot  be  registered  to  the  preoperative  CT  volume  directly.  What  can  be  done  is  to  find  a  subvolume  of  the  preoperative 
CT  volume  corresponding  to  the  dashed  box,  and  then  stack  the  tissue  of  the  subvolume  together  to  generate  a  synthetic 
CT  fluoroscopy  image.  If  the  real  and  the  synthetic  CT  fluoroscopy  images  match  each  other,  the  registration  is  done. 
The  following  equation  (1)  describes  the  relationship  between  the  2D  CT  fluoroscopy  image  /  and  its  corresponding  3D 
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subvolume  Vin  preoperative  CT: 

^  h/2 


z^-hn 


(1) 


where  n  represents  the  noise,  f  represents  a  deformable  transformation  and  h  is  the  slice  thickness  of  the  CT  fluoroscopy 
image.  The  integral  is  to  project  the  subvolume  on  a  plane  to  simulate  the  CT  fluoroscopy  image.  It  can  be  realized  by 
tri-linear  interpolation. 

The  deformation  of  the  lung  tissue  can  be  very  complicated.  However,  if  the  region  to  be  registered  is  small,  the 
deformation  can  be  approximated  using  3D  affine  transformation,  which  is  a  3  by  4  parameter  matrix  represented  by  f 
in  Fig.  5.  Equation  (2)  shows  a  basic  objective  function  to  be  minimized.  It  is  a  weighted  SSD  between  the  CT 
fluoroscopy  region  and  the  synthetic  region. 


1 

-  jv (  f  {x,  y,  z,  fi)  )dz  -I{x,y) 

”  - _ u  I  'i 


(2) 


where  |i  is  a  vector  of  twelve  parameters  of  the  affine  transformation  to  be  estimated;  Wa  is  a  Gaussian  window  function 
which  gives  the  central  pixels  a  larger  weight.  As  a  result,  the  outer  pixels  that  can  not  be  well  represented  by  the  affine 
transformation  will  not  have  a  large  effect  on  the  objective  function.  Wb  is  another  weighting  function  with  gives  the 
brighter  pixels  a  larger  weight  as  described  in  Equation  3. 


Wj  =  max 


hn  ^ 

-  \\{f{x,y,z,\i))dz,  ICx.y) 


(3) 


The  purpose  of  Wb  is  to  emphasize  the  bright  texture  generated  by  the  blood  vessels  and  bronchi.  If  both  of  the  pixels 
being  compared  are  dark,  their  effect  on  the  objective  function  will  be  minimal.  The  dark  pixels  usually  correspond  to 
the  parenchyma  of  the  lung,  which  can  only  generate  noise  to  the  SSD. 


Normally,  SSD  is  only  used  for  the  intra-modality  registration.  It  can  be  shown  that  SSD  is  the  optimum  measure  when 
two  images  only  differ  by  Gaussian  noise  [10].  In  this  tracking  problem,  the  image  regions  being  compared  come  from 
fte  sme  machine  but  two  different  imaging  methods  -  the  standard  CT  and  the  CT  fluoroscopy,  which  results  in  an 
intensity  difference  between  the  two  regions.  If  this  intensity  difference  is  big,  SSD  misregistration  will  happen. 
Although  there  is  no  simple  relationship  to  map  the  intensity  from  one  imaging  method  to  the  other,  the  effect  of  the 
intensity  difference  can  be  reduced  by  subtracting  the  mean  of  the  intensity  from  both  regions.  The  revised  objective 
function  is  then  the  following; 


(x,y)^n 


1 

-  JV(f(x,y,z,p))cfe-C^ 

”  _ _ U  /  T 


L"  z=-ht2 


(4) 


where  Q  and  Ci  are  the  intensity  means  of  the  synthetic  region  and  the  CT  fluoroscopy  region  respectively.  They  are 
given  in  equations  5  and  6,  in  which  N  is  the  number  of  pixels  inside  the  region  of  interest  Q. 

<^v=—  2  T  \^ihx,y,z,\i))dz  (5) 

^  ix,y)^n  n 

(6) 

Technically,  the  mean  of  the  synthetic  region  is  a  function  of  the  unknowns  of  the  affine  transformation.  However,  the 
SSD  function  is  solved  iteratively.  Since  the  step  size  is  very  small  between  two  consecutive  iterations,  the  change  of 
mean  is  tiny  and  has  little  effect  on  the  solution.  Therefore,  the  intensity  mean  of  the  synthetic  region  is  treated  as  a 
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constant.  It  was  found  that  the  subtraction  of  the  mean  intensity  from  the  original  intensity  is  very  effective  in  improving 
the  robustness  of  the  algorithm. 

The  reason  to  use  SSD  is  to  replace  the  image  registration  problem  with  non-linear  minimization.  Using  the  standard 
optimization  methods,  the  tracking  algorithm  can  be  veiy  efficient.  There  is  no  need  to  evaluate  the  entire  objective 
function  to  calculate  its  derivatives.  The  derivatives  can  be  obtained  from  the  image  gradients  calculated  offline  from  the 
pre-operative  CT  volume. 

2.3  Similarity  voting 

Both  multi-start  and  importance  sampling  strategies  are  used  to  avoid  local  minima,  which  result  in  multiple  candidate 
transformations  of  the  CT  fluoroscopy  image  region.  Many  of  these  transformations  are  false  results  caused  by  local 
minima  of  the  SSD  residual  error.  Even  if  the  global  minimum  is  found,  it  may  not  be  the  true  transformation  due  to  the 
effect  of  the  image  noise.  In  response  to  these  problems,  similarity  voting  is  used  to  distinguish  the  true  transformation 
from  the  false  transformations.  The  following  measures  are  used  to  evaluate  the  similarity  between  the  CT  fluoroscopy 
image  region  and  the  synthetic  region. 

a)  Residual  error  of  the  pixels  inside  the  region 

b)  Residual  error  of  the  pixels  surrounding  the  region 

c)  Residual  error  of  the  bright  pixels  inside  the  region 

d)  Axis  orthogonality 

In  the  above  similarity  measures,  (a)  is  the  value  of  the  SSD  objective  function  and  (b)  gives  the  “global”  information  of 
the  region.  The  bright  pixels  in  (c)  are  segmented  using  automatic  thresholding  [11].  They  are  emphasized  because  the 
blood  vessels  and  bronchi  appear  bright  in  the  image,  which  form  most  of  the  useful  texture.  The  assumption  for  (d)  is 
that  the  deformation  is  small,  so  the  principle  axes  are  nearly  perpendicular  to  each  other  after  the  affine  transformation. 
Although  none  of  the  four  similarity  measures  is  sufficient  to  independently  determine  the  true  transformation,  the  true 
transformation  should  generally  have  larger  values  of  the  similarity  measures  compared  to  the  other  candidate 
transformations.  If  the  candidate  transformations  are  ranked  in  terms  of  overall  evaluations  from  all  the  similarity 
measures,  the  true  transformation  is  very  likely  to  have  the  best  overall  ranking.  As  shown  in  Fig.  6,  the  candidate 
transformations  are  evaluated  and  ranked  for  each  similarity  measure.  The  four  rankings  for  each  candidate  are  summed. 
The  candidate  transformation  with  the  smallest  sum  is  selected.  With  enough  similarity  measures,  the  similarity  voting 
can  be  robust  in  picking  out  the  true  motion  vector.  In  addition,  it  avoids  using  arbitrarily  set  thresholds,  which  is 
problem  specific  and  prone  to  failure. 

2.4  Kalman  filtering 

The  Kalman  filter  is  a  classical  tool  that  produces  estimates  optimal  in  the  least-squares  sense  of  the  state  of  a  dynamic 
system  from  noisy  measurements  and  an  uncertain  model  of  the  system  dynamics.  The  use  of  predictors  like  Kalman 
filter  can  reduce  computational  cost  by  only  processing  a  certain  area  of  the  image,  and  the  noise  outside  that  area  does 
not  influence  the  processing  [12].  The  price  paid  for  the  efficiency  is  the  increased  possibility  of  dropping  the  true 
motion  vector.  Using  the  multi-start  strategy  mentioned  above,  the  dropping  problem  can  be  alleviated.  In  this  research, 
the  dynamic  system  equation  of  the  Kalman  filter  is  chosen  to  be  a  constant  velocity  update.  Although  it  is  possible  to 
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Fig.7  Respiratory  motion  simulator  Fig.8  Motion  tracking  test  of  respiratory  motion  simulator 


use  a  more  complicated  motion  model  to  boost  the  Kalman  filter’s  performance,  the  disadvantage  is  that  the  tuned 
Kalman  filter  is  effective  only  for  a  narrow  class  of  motions.  Given  that  the  motion  of  the  target  lesion  is  not  limited  to 
respiratory  motion,  and  the  patient  can  hold  the  breath  during  the  needle  insertion,  it  may  not  be  worthwhile  to  employ  a 
complicated  motion  model. 


3.  EXPERIMENTAL  RESULTS 

Three  experiments  were  carried  out  using  both  synthetic  and  real  CT  fluoroscopy  images  (Siemens  Somatom  Volume 
Zoom  CT).  The  synthetic  CT  fluoroscopy  image  was  generated  from  a  pre-operative  CT  volume  of  the  lung  region  with 
2.0  mm  slice  thickness  and  0.72  mm  pixel  size.  As  a  result,  the  exact  match  of  the  CT  fluoroscopy  image  region  is 
known.  Clustering  analysis  of  the  SSD  objective  function  was  performed.  It  was  observed  that,  for  the  objective  function 
to  converge,  the  starting  point  had  to  be  within  ±2mm  for  translation  and  within  ±20°  for  each  of  the  three  rotation 
angles.  A  similar  experiment  was  made  using  two  different  CT  volumes  of  the  same  patient.  With  one  volume  as  the  pre¬ 
operative  CT  volume,  and  the  other  one  to  generate  synthetic  CT  fluoroscopy  images,  similar  results  were  obtained. 

The  second  experiment  was  performed  with  CT  fluoroscopy  images  from  a  respiratory  motion  simulator.  As  shown  in 
Fig.  7,  the  target  organ  had  one  degree  of  freedom  and  was  moved  by  a  motor  in  the  cranio-caudal  direction.  After  a  pre- 
operative  volumetric  scan  was  obtained,  the  data  was  saved  in  DICOM  format  with  512  x  512  resolution,  1mm  slice 
thickness  and  0.74  mm  pixel  size.  The  motor  of  the  simulator  was  aligned  with  the  CT  gantry  such  that  the  motion  of  the 
phantom  was  perpendicular  to  the  imaging  plane  (Fig.  8).  Since  the  organ  phantom  was  rigid,  its  motion  curve  was  the 
same  as  that  of  Ae  motor.  A  sequence  of  CT  fluoroscopy  images  were  taken  at  about  6Hz  and  saved  in  the  DICOM 
format.  The  resolution  of  the  CT  fluoroscopy  image  was  256  x  256,  with  10mm  slice  thickness  and  1.48  mm  pixel  size. 
A  10mm  radius  region  was  selected  on  the  first  frame  of  the  CT  fluoroscopy  image.  The  motion  tracking  was  initialized. 
Fig.  9  shows  the  motion  tracking  results  compared  to  the  known  motion  curve  of  the  motor  at  two  different  motion 
modes.  The  major  motion  in  the  Z  direction  (perpendicular  to  the  imaging  plane)  was  very  consistent  with  the  motion 
curve  of  the  motor.  The  maximum  displacement  in  X  and  Y  directions  was  under  0.8  mm  and  treated  as  noise.  The 
average  position  error  was  under  1mm,  if  the  delay  in  time  is  ignored.  As  might  be  expected,  the  tracking  algorithm 
tended  to  have  a  small  position  error  when  the  target’s  motion  was  slow  or  close  to  zero. 


Fig.9  Experimental  results  of  simulator  test.  The  left  and  right  curves  show  the  displacement  of  the  target  organ  at  two  different 
motions.  The  displacement  in  Z  direction  is  compared  to  the  ground  truth  of  the  motion  curve  of  the  phantom. 
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Fig.  10  Swine  study  Fig.  1 1  CT  fluoroscopy  image  of  Swine  study 


The  third  experiment  was  a  swine  study  (Fig.  10).  This  study  was  done  under  an  approved  animal  care  protocol.  Most  of 
the  parameters  were  the  same  as  the  second  experiment  except  that  the  pixel  size  was  0.47  mm  for  the  preoperative  CT 
image  and  1.48  mm  for  the  CT  fluoroscopy  image.  The  respiratory  rate  was  set  at  1 1  cycles  per  minute  using  a  ventilator. 
Iiistead  of  using  the  frame  grabber,  the  CT  fluoroscopy  images  were  obtained  from  the  DICOM  files  saved  by  the  CT. 
Since  the  CT  machine  can  only  save  about  the  last  twenty  frames  of  CT  fluoroscopy  images,  only  a  portion  of  the 
respiratory  cycle  was  recorded.  Fig.  12  shows  the  trajectories  of  two  different  locations  of  the  CT  fluoroscopy  image 
during  the  respiratory  cycle.  The  curves  on  the  left  and  right  correspond  to  the  regions  1  and  2  in  Fig.  11  respectively. 
While  the  motion  patterns  are  similar  in  some  respects,  there  are  enough  differences  to  note  that  different  regions  in  the 
lung  (even  regions  that  are  close  by)  will  move  with  different  patterns.  With  a  2G  Hz  Pentium  4  CPU,  the  frequency  of 
the  tracking  algorithm  was  between  2Hz  and  3Hz.  The  algorithm  worked  for  most  of  the  lung  region.  For  some  regions 
with  little  texture,  the  SSD  algorithm  failed,  most  likely  because  it  did  not  have  enough  information  to  uniquely 
determine  the  match  of  the  CT  fluoroscopy  image  region  in  the  CT  volume. 


Fig.l2  Motion  estimation.  The  pictures  on  the  left  /  right  shows  the  motion  curves  of  region  1  /  2  in  Fig.  1 1. 
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4.  DISCUSSION 

The  basic  assumption  of  region-based  tracking  methods  is  that  the  image  pixels  inside  the  tracking  region  undergo  a 
single  affine  transformation.  This  assumption  is  often  violated  by  the  lung  region.  The  disadvantage  of  such  an  approach 
is  the  lack  of  resolution  -  a  fixed  region  size  means  that  the  approach  cannot  adapt  to  the  underlying  data  [13].  One  way 
to  improve  the  algorithm  is  to  use  more  sophisticated  deformable  model  such  as  spline  deformation  rather  than  the  affine 
transformation.  As  a  result,  the  analysis  region  can  be  bigger,  which  should  also  make  the  algorithm  more  robust. 
Another  problem  that  may  affect  the  accuracy  is  that,  instead  of  tracking  the  target  lesion  directly,  the  algorithm  tracks 
the  region  inside  the  CT  fluoroscopy  imaging  plane.  If  the  local  rigidity  assumption  of  tracking  region  and  the  target 
lesion  is  violated,  more  error  will  be  introduced. 

Compared  to  the  conventional  motion  tracking  problem  in  computer  vision,  the  tracking  problem  of  this  paper  has  the 
following  properties: 

(1)  since  the  CT  fluoroscopy  image  region  changes  over  time,  the  matching  template  is  not  fixed,  and  the  efficient 
tracking  algorithm  presented  in  [2]  cannot  apply; 

(2)  there  is  no  illumination  change  in  the  problem.  Instead,  the  difference  between  CT  images  and  CT  fluoroscopy 

images  needs  to  be  adjusted; 

(3)  the  algorithm  does  not  need  to  deal  with  the  object  occlusion.  The  lack  of  texture  and  large  deformation  are  the 

major  reasons  to  lose  the  tracking. 

Experimental  results  showed  that  the  algorithm  worked  reasonably  well  in  most  of  the  lung  area.  At  some  locations 
where  there  was  not  much  texture,  the  algorithm  could  not  track  the  target.  This  could  be  mitigated  by  selecting  a  region 
close  to  the  target  lesion  that  the  algorithm  is  able  to  track,  and  use  that  region  to  infer  the  location  of  the  target.  In 
addition,  the  algorithm  can  be  combined  with  other  methods  to  increase  its  robustness.  For  instance,  we  can  track  the 
motion  of  the  chest  wall  and  try  to  correlate  this  motion  with  the  motion  of  the  target  lesion.  In  case  the  SSD  algorithm 
fails,  the  motion  of  the  chest  wall  may  be  able  to  be  used  to  estimate  the  motion  of  the  target  lesion. 

In  order  to  evaluate  the  algorithm,  it  is  desirable  to  use  a  second  tracking  method  to  obtain  the  ground  truth  of  the  target 
motion.  However,  this  is  often  impractical.  For  example,  although  the  magnetic  tracker  can  track  the  motion  of  the 
internal  organ,  its  performance  is  very  poor  in  a  CT  scanner.  Currently,  we  are  trying  to  get  a  4D  CT  dataset  [14]  to 
evaluate  the  algorithm. 

Our  future  work  will  focus  on  improving  the  robustness  of  the  algorithm.  A  post-failure  recovery  strategy  should  be 
established.  Chest  wall  motion  tracking  and  its  correspondence  to  the  target  lesion  will  be  investigated.  Additional 
animal  studies  are  also  planned. 


5.  CONCLUSION 

This  paper  presented  a  new  algorithm  to  track  the  motion  of  pulmonary  lesions  using  CT  fluoroscopy.  Initial 
experimental  results  showed  that  the  algorithm  worked  well  with  a  respiratory  motion  simulator  and  has  reasonable 
performance  in  a  swine  study.  Future  experiments  are  planned  to  incorporate  the  algorithm  with  the  robotically  assisted 
lung  biopsy  system. 

To  the  best  of  the  authors’  knowledge,  there  are  no  other  published  methods  that  are  non-invasive  and  can  track 
pulmonary  lesions  at  any  location  of  the  lung.  The  algorithm  is  automated  and  can  run  in  nearly  real-time.  Although  the 
current  speed  of  the  algorithm  (2-3Hz)  is  not  fast  enough  to  keep  up  with  the  CT  fluoroscopy  update  rate  (6  Hz),  with  a 
faster  CPU  and  the  optimization  of  the  algorithm,  it  is  hoped  that  the  algorithm  will  work  in  real-time. 
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